[2023-10-12 19:55:50,445][43579] Saving configuration to ./train_atari/atari_krull_APPO/config.json... [2023-10-12 19:55:50,762][43579] Rollout worker 0 uses device cpu [2023-10-12 19:55:50,763][43579] Rollout worker 1 uses device cpu [2023-10-12 19:55:50,764][43579] Rollout worker 2 uses device cpu [2023-10-12 19:55:50,764][43579] Rollout worker 3 uses device cpu [2023-10-12 19:55:50,765][43579] Rollout worker 4 uses device cpu [2023-10-12 19:55:50,765][43579] Rollout worker 5 uses device cpu [2023-10-12 19:55:50,766][43579] Rollout worker 6 uses device cpu [2023-10-12 19:55:50,766][43579] Rollout worker 7 uses device cpu [2023-10-12 19:55:50,767][43579] Rollout worker 8 uses device cpu [2023-10-12 19:55:50,767][43579] Rollout worker 9 uses device cpu [2023-10-12 19:55:50,767][43579] Rollout worker 10 uses device cpu [2023-10-12 19:55:50,768][43579] Rollout worker 11 uses device cpu [2023-10-12 19:55:50,768][43579] Rollout worker 12 uses device cpu [2023-10-12 19:55:50,769][43579] Rollout worker 13 uses device cpu [2023-10-12 19:55:50,769][43579] Rollout worker 14 uses device cpu [2023-10-12 19:55:50,769][43579] Rollout worker 15 uses device cpu [2023-10-12 19:55:51,052][43579] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-12 19:55:51,053][43579] InferenceWorker_p0-w0: min num requests: 2 [2023-10-12 19:55:51,056][43579] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-12 19:55:51,056][43579] InferenceWorker_p1-w0: min num requests: 2 [2023-10-12 19:55:51,102][43579] Starting all processes... [2023-10-12 19:55:51,102][43579] Starting process learner_proc0 [2023-10-12 19:55:52,794][43579] Starting process learner_proc1 [2023-10-12 19:55:52,797][44518] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-12 19:55:52,797][44518] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-10-12 19:55:52,816][44518] Num visible devices: 1 [2023-10-12 19:55:52,835][44518] Setting fixed seed 1234 [2023-10-12 19:55:52,837][44518] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-12 19:55:52,837][44518] Initializing actor-critic model on device cuda:0 [2023-10-12 19:55:52,837][44518] RunningMeanStd input shape: (4, 84, 84) [2023-10-12 19:55:52,838][44518] RunningMeanStd input shape: (1,) [2023-10-12 19:55:52,849][44518] ConvEncoder: input_channels=4 [2023-10-12 19:55:53,004][44518] Conv encoder output size: 512 [2023-10-12 19:55:53,006][44518] Created Actor Critic model with architecture: [2023-10-12 19:55:53,006][44518] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=18, bias=True) ) ) [2023-10-12 19:55:53,594][44518] Using optimizer [2023-10-12 19:55:53,595][44518] No checkpoints found [2023-10-12 19:55:53,597][44518] Did not load from checkpoint, starting from scratch! [2023-10-12 19:55:53,597][44518] Initialized policy 0 weights for model version 0 [2023-10-12 19:55:53,599][44518] LearnerWorker_p0 finished initialization! [2023-10-12 19:55:53,599][44518] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-12 19:55:54,576][43579] Starting all processes... [2023-10-12 19:55:54,579][44583] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-12 19:55:54,579][44583] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-10-12 19:55:54,584][43579] Starting process inference_proc0-0 [2023-10-12 19:55:54,584][43579] Starting process inference_proc1-0 [2023-10-12 19:55:54,585][43579] Starting process rollout_proc0 [2023-10-12 19:55:54,598][44583] Num visible devices: 1 [2023-10-12 19:55:54,585][43579] Starting process rollout_proc1 [2023-10-12 19:55:54,585][43579] Starting process rollout_proc2 [2023-10-12 19:55:54,616][44583] Setting fixed seed 1234 [2023-10-12 19:55:54,586][43579] Starting process rollout_proc3 [2023-10-12 19:55:54,617][44583] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-12 19:55:54,617][44583] Initializing actor-critic model on device cuda:0 [2023-10-12 19:55:54,618][44583] RunningMeanStd input shape: (4, 84, 84) [2023-10-12 19:55:54,618][44583] RunningMeanStd input shape: (1,) [2023-10-12 19:55:54,586][43579] Starting process rollout_proc4 [2023-10-12 19:55:54,586][43579] Starting process rollout_proc5 [2023-10-12 19:55:54,591][43579] Starting process rollout_proc6 [2023-10-12 19:55:54,596][43579] Starting process rollout_proc7 [2023-10-12 19:55:54,597][43579] Starting process rollout_proc8 [2023-10-12 19:55:54,600][43579] Starting process rollout_proc9 [2023-10-12 19:55:54,630][44583] ConvEncoder: input_channels=4 [2023-10-12 19:55:54,602][43579] Starting process rollout_proc10 [2023-10-12 19:55:54,603][43579] Starting process rollout_proc11 [2023-10-12 19:55:54,603][43579] Starting process rollout_proc12 [2023-10-12 19:55:54,603][43579] Starting process rollout_proc13 [2023-10-12 19:55:54,881][44583] Conv encoder output size: 512 [2023-10-12 19:55:54,884][44583] Created Actor Critic model with architecture: [2023-10-12 19:55:54,884][44583] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=18, bias=True) ) ) [2023-10-12 19:55:55,802][44583] Using optimizer [2023-10-12 19:55:55,803][44583] No checkpoints found [2023-10-12 19:55:55,803][44583] Did not load from checkpoint, starting from scratch! [2023-10-12 19:55:55,803][44583] Initialized policy 1 weights for model version 0 [2023-10-12 19:55:55,805][44583] LearnerWorker_p1 finished initialization! [2023-10-12 19:55:55,806][44583] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-12 19:55:56,755][43579] Starting process rollout_proc14 [2023-10-12 19:55:56,759][43579] Starting process rollout_proc15 [2023-10-12 19:55:56,762][44991] Worker 1 uses CPU cores [2, 3] [2023-10-12 19:55:56,764][44993] Worker 0 uses CPU cores [0, 1] [2023-10-12 19:55:56,770][44995] Worker 2 uses CPU cores [4, 5] [2023-10-12 19:55:56,844][45004] Worker 11 uses CPU cores [22, 23] [2023-10-12 19:55:57,042][45003] Worker 10 uses CPU cores [20, 21] [2023-10-12 19:55:57,110][45002] Worker 9 uses CPU cores [18, 19] [2023-10-12 19:55:57,142][45000] Worker 7 uses CPU cores [14, 15] [2023-10-12 19:55:57,151][45001] Worker 8 uses CPU cores [16, 17] [2023-10-12 19:55:57,220][44959] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-12 19:55:57,220][44959] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-10-12 19:55:57,239][44959] Num visible devices: 1 [2023-10-12 19:55:57,331][44958] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-12 19:55:57,331][44958] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-10-12 19:55:57,346][45005] Worker 13 uses CPU cores [26, 27] [2023-10-12 19:55:57,350][44958] Num visible devices: 1 [2023-10-12 19:55:57,370][44997] Worker 4 uses CPU cores [8, 9] [2023-10-12 19:55:57,398][44998] Worker 5 uses CPU cores [10, 11] [2023-10-12 19:55:57,450][44999] Worker 6 uses CPU cores [12, 13] [2023-10-12 19:55:57,588][44996] Worker 3 uses CPU cores [6, 7] [2023-10-12 19:55:57,599][45006] Worker 12 uses CPU cores [24, 25] [2023-10-12 19:55:57,848][44959] RunningMeanStd input shape: (4, 84, 84) [2023-10-12 19:55:57,848][44959] RunningMeanStd input shape: (1,) [2023-10-12 19:55:57,859][44959] ConvEncoder: input_channels=4 [2023-10-12 19:55:57,959][44958] RunningMeanStd input shape: (4, 84, 84) [2023-10-12 19:55:57,959][44958] RunningMeanStd input shape: (1,) [2023-10-12 19:55:57,961][44959] Conv encoder output size: 512 [2023-10-12 19:55:57,970][44958] ConvEncoder: input_channels=4 [2023-10-12 19:55:58,069][44958] Conv encoder output size: 512 [2023-10-12 19:55:58,664][45783] Worker 14 uses CPU cores [28, 29] [2023-10-12 19:55:58,674][43579] Inference worker 1-0 is ready! [2023-10-12 19:55:58,675][43579] Inference worker 0-0 is ready! [2023-10-12 19:55:58,675][45784] Worker 15 uses CPU cores [30, 31] [2023-10-12 19:55:58,676][43579] All inference workers are ready! Signal rollout workers to start! [2023-10-12 19:55:58,677][44999] EnvRunner 6-0 uses policy 0 [2023-10-12 19:55:58,677][45000] EnvRunner 7-0 uses policy 1 [2023-10-12 19:55:58,677][44993] EnvRunner 0-0 uses policy 0 [2023-10-12 19:55:58,677][45005] EnvRunner 13-0 uses policy 1 [2023-10-12 19:55:58,677][44991] EnvRunner 1-0 uses policy 1 [2023-10-12 19:55:58,677][44996] EnvRunner 3-0 uses policy 1 [2023-10-12 19:55:58,677][44995] EnvRunner 2-0 uses policy 0 [2023-10-12 19:55:58,677][45004] EnvRunner 11-0 uses policy 1 [2023-10-12 19:55:58,677][45001] EnvRunner 8-0 uses policy 0 [2023-10-12 19:55:58,677][44998] EnvRunner 5-0 uses policy 1 [2023-10-12 19:55:58,677][45002] EnvRunner 9-0 uses policy 1 [2023-10-12 19:55:58,677][45006] EnvRunner 12-0 uses policy 0 [2023-10-12 19:55:58,678][45003] EnvRunner 10-0 uses policy 0 [2023-10-12 19:55:58,678][44997] EnvRunner 4-0 uses policy 0 [2023-10-12 19:55:58,677][43579] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-12 19:55:58,851][45783] EnvRunner 14-0 uses policy 0 [2023-10-12 19:55:58,895][45784] EnvRunner 15-0 uses policy 1 [2023-10-12 19:56:01,040][43579] Heartbeat connected on Batcher_0 [2023-10-12 19:56:01,043][43579] Heartbeat connected on LearnerWorker_p0 [2023-10-12 19:56:01,046][43579] Heartbeat connected on Batcher_1 [2023-10-12 19:56:01,048][43579] Heartbeat connected on LearnerWorker_p1 [2023-10-12 19:56:01,056][43579] Heartbeat connected on InferenceWorker_p0-w0 [2023-10-12 19:56:01,062][43579] Heartbeat connected on InferenceWorker_p1-w0 [2023-10-12 19:56:01,063][43579] Heartbeat connected on RolloutWorker_w0 [2023-10-12 19:56:01,063][43579] Heartbeat connected on RolloutWorker_w1 [2023-10-12 19:56:01,069][43579] Heartbeat connected on RolloutWorker_w2 [2023-10-12 19:56:01,071][43579] Heartbeat connected on RolloutWorker_w4 [2023-10-12 19:56:01,072][43579] Heartbeat connected on RolloutWorker_w3 [2023-10-12 19:56:01,076][43579] Heartbeat connected on RolloutWorker_w5 [2023-10-12 19:56:01,078][43579] Heartbeat connected on RolloutWorker_w6 [2023-10-12 19:56:01,082][43579] Heartbeat connected on RolloutWorker_w7 [2023-10-12 19:56:01,083][43579] Heartbeat connected on RolloutWorker_w8 [2023-10-12 19:56:01,087][43579] Heartbeat connected on RolloutWorker_w10 [2023-10-12 19:56:01,089][43579] Heartbeat connected on RolloutWorker_w9 [2023-10-12 19:56:01,092][43579] Heartbeat connected on RolloutWorker_w12 [2023-10-12 19:56:01,093][43579] Heartbeat connected on RolloutWorker_w11 [2023-10-12 19:56:01,095][43579] Heartbeat connected on RolloutWorker_w13 [2023-10-12 19:56:01,099][43579] Heartbeat connected on RolloutWorker_w14 [2023-10-12 19:56:01,102][43579] Heartbeat connected on RolloutWorker_w15 [2023-10-12 19:56:01,442][43579] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 679.2, 1: 655.3. Samples: 3690. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-12 19:56:01,443][43579] Avg episode reward: [(1, '1.000')] [2023-10-12 19:56:06,443][43579] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 996.2, 1: 984.9. Samples: 15384. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-12 19:56:06,444][43579] Avg episode reward: [(0, '27.800'), (1, '38.875')] [2023-10-12 19:56:08,822][44959] Updated weights for policy 1, policy_version 10 (0.0009) [2023-10-12 19:56:08,930][44958] Updated weights for policy 0, policy_version 10 (0.0009) [2023-10-12 19:56:09,178][44959] Updated weights for policy 1, policy_version 20 (0.0009) [2023-10-12 19:56:09,292][44958] Updated weights for policy 0, policy_version 20 (0.0009) [2023-10-12 19:56:09,538][44959] Updated weights for policy 1, policy_version 30 (0.0009) [2023-10-12 19:56:09,665][44958] Updated weights for policy 0, policy_version 30 (0.0008) [2023-10-12 19:56:11,443][43579] Fps is (10 sec: 6553.5, 60 sec: 5133.9, 300 sec: 5133.9). Total num frames: 65536. Throughput: 0: 1218.9, 1: 1220.0. Samples: 31134. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 19:56:11,444][43579] Avg episode reward: [(0, '44.176'), (1, '42.941')] [2023-10-12 19:56:12,221][44959] Updated weights for policy 1, policy_version 40 (0.0009) [2023-10-12 19:56:12,415][44958] Updated weights for policy 0, policy_version 40 (0.0009) [2023-10-12 19:56:12,591][44959] Updated weights for policy 1, policy_version 50 (0.0008) [2023-10-12 19:56:12,777][44958] Updated weights for policy 0, policy_version 50 (0.0008) [2023-10-12 19:56:12,953][44959] Updated weights for policy 1, policy_version 60 (0.0009) [2023-10-12 19:56:13,145][44958] Updated weights for policy 0, policy_version 60 (0.0008) [2023-10-12 19:56:16,433][44959] Updated weights for policy 1, policy_version 70 (0.0008) [2023-10-12 19:56:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 7378.0, 300 sec: 7378.0). Total num frames: 131072. Throughput: 0: 1450.9, 1: 1444.2. Samples: 51432. Policy #0 lag: (min: 33.0, avg: 33.0, max: 33.0) [2023-10-12 19:56:16,443][43579] Avg episode reward: [(0, '46.440'), (1, '38.556')] [2023-10-12 19:56:16,793][44959] Updated weights for policy 1, policy_version 80 (0.0008) [2023-10-12 19:56:16,839][44958] Updated weights for policy 0, policy_version 70 (0.0008) [2023-10-12 19:56:17,155][44959] Updated weights for policy 1, policy_version 90 (0.0010) [2023-10-12 19:56:17,213][44958] Updated weights for policy 0, policy_version 80 (0.0010) [2023-10-12 19:56:17,573][44958] Updated weights for policy 0, policy_version 90 (0.0007) [2023-10-12 19:56:20,940][44959] Updated weights for policy 1, policy_version 100 (0.0007) [2023-10-12 19:56:21,310][44959] Updated weights for policy 1, policy_version 110 (0.0008) [2023-10-12 19:56:21,340][44958] Updated weights for policy 0, policy_version 100 (0.0010) [2023-10-12 19:56:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 8636.2, 300 sec: 8636.2). Total num frames: 196608. Throughput: 0: 1327.2, 1: 1327.1. Samples: 60426. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 19:56:21,444][43579] Avg episode reward: [(0, '52.439'), (1, '45.986')] [2023-10-12 19:56:21,679][44959] Updated weights for policy 1, policy_version 120 (0.0009) [2023-10-12 19:56:21,706][44958] Updated weights for policy 0, policy_version 110 (0.0007) [2023-10-12 19:56:22,087][44958] Updated weights for policy 0, policy_version 120 (0.0009) [2023-10-12 19:56:26,095][44959] Updated weights for policy 1, policy_version 130 (0.0010) [2023-10-12 19:56:26,377][44958] Updated weights for policy 0, policy_version 130 (0.0008) [2023-10-12 19:56:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 9441.5, 300 sec: 9441.5). Total num frames: 262144. Throughput: 0: 1440.9, 1: 1445.2. Samples: 80132. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-12 19:56:26,443][43579] Avg episode reward: [(0, '56.477'), (1, '50.287')] [2023-10-12 19:56:26,467][44959] Updated weights for policy 1, policy_version 140 (0.0009) [2023-10-12 19:56:26,743][44958] Updated weights for policy 0, policy_version 140 (0.0008) [2023-10-12 19:56:26,829][44959] Updated weights for policy 1, policy_version 150 (0.0008) [2023-10-12 19:56:27,117][44958] Updated weights for policy 0, policy_version 150 (0.0007) [2023-10-12 19:56:27,190][44583] Saving new best policy, reward=50.287! [2023-10-12 19:56:27,192][44959] Updated weights for policy 1, policy_version 160 (0.0008) [2023-10-12 19:56:27,478][44518] Saving new best policy, reward=56.477! [2023-10-12 19:56:27,480][44958] Updated weights for policy 0, policy_version 160 (0.0008) [2023-10-12 19:56:31,330][44959] Updated weights for policy 1, policy_version 170 (0.0008) [2023-10-12 19:56:31,427][44958] Updated weights for policy 0, policy_version 170 (0.0007) [2023-10-12 19:56:31,443][43579] Fps is (10 sec: 13107.5, 60 sec: 10000.8, 300 sec: 10000.8). Total num frames: 327680. Throughput: 0: 1529.4, 1: 1532.1. Samples: 100308. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-12 19:56:31,443][43579] Avg episode reward: [(0, '60.390'), (1, '55.800')] [2023-10-12 19:56:31,692][44959] Updated weights for policy 1, policy_version 180 (0.0007) [2023-10-12 19:56:31,802][44958] Updated weights for policy 0, policy_version 180 (0.0008) [2023-10-12 19:56:32,055][44959] Updated weights for policy 1, policy_version 190 (0.0007) [2023-10-12 19:56:32,122][44583] Saving new best policy, reward=55.800! [2023-10-12 19:56:32,171][44958] Updated weights for policy 0, policy_version 190 (0.0009) [2023-10-12 19:56:32,242][44518] Saving new best policy, reward=60.390! [2023-10-12 19:56:36,279][44959] Updated weights for policy 1, policy_version 200 (0.0007) [2023-10-12 19:56:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 10412.1, 300 sec: 10412.1). Total num frames: 393216. Throughput: 0: 1443.7, 1: 1447.9. Samples: 109200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 19:56:36,443][43579] Avg episode reward: [(0, '66.130'), (1, '61.600')] [2023-10-12 19:56:36,451][44958] Updated weights for policy 0, policy_version 200 (0.0007) [2023-10-12 19:56:36,641][44959] Updated weights for policy 1, policy_version 210 (0.0007) [2023-10-12 19:56:36,818][44958] Updated weights for policy 0, policy_version 210 (0.0007) [2023-10-12 19:56:37,015][44959] Updated weights for policy 1, policy_version 220 (0.0008) [2023-10-12 19:56:37,161][44583] Saving new best policy, reward=61.600! [2023-10-12 19:56:37,185][44958] Updated weights for policy 0, policy_version 220 (0.0010) [2023-10-12 19:56:37,334][44518] Saving new best policy, reward=66.130! [2023-10-12 19:56:41,002][44959] Updated weights for policy 1, policy_version 230 (0.0008) [2023-10-12 19:56:41,355][44959] Updated weights for policy 1, policy_version 240 (0.0009) [2023-10-12 19:56:41,381][44958] Updated weights for policy 0, policy_version 230 (0.0010) [2023-10-12 19:56:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 10727.2, 300 sec: 10727.2). Total num frames: 458752. Throughput: 0: 1507.2, 1: 1518.1. Samples: 129376. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 19:56:41,443][43579] Avg episode reward: [(0, '69.250'), (1, '70.040')] [2023-10-12 19:56:41,715][44959] Updated weights for policy 1, policy_version 250 (0.0010) [2023-10-12 19:56:41,754][44958] Updated weights for policy 0, policy_version 240 (0.0010) [2023-10-12 19:56:41,930][44583] Saving new best policy, reward=70.040! [2023-10-12 19:56:42,129][44958] Updated weights for policy 0, policy_version 250 (0.0010) [2023-10-12 19:56:42,344][44518] Saving new best policy, reward=69.250! [2023-10-12 19:56:46,012][44959] Updated weights for policy 1, policy_version 260 (0.0009) [2023-10-12 19:56:46,244][44958] Updated weights for policy 0, policy_version 260 (0.0011) [2023-10-12 19:56:46,369][44959] Updated weights for policy 1, policy_version 270 (0.0009) [2023-10-12 19:56:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 10976.4, 300 sec: 10976.4). Total num frames: 524288. Throughput: 0: 1618.7, 1: 1624.5. Samples: 149636. Policy #0 lag: (min: 4.0, avg: 7.8, max: 36.0) [2023-10-12 19:56:46,443][43579] Avg episode reward: [(0, '72.450'), (1, '77.940')] [2023-10-12 19:56:46,614][44958] Updated weights for policy 0, policy_version 270 (0.0009) [2023-10-12 19:56:46,731][44959] Updated weights for policy 1, policy_version 280 (0.0009) [2023-10-12 19:56:46,986][44958] Updated weights for policy 0, policy_version 280 (0.0008) [2023-10-12 19:56:47,023][44583] Saving new best policy, reward=77.940! [2023-10-12 19:56:47,280][44518] Saving new best policy, reward=72.450! [2023-10-12 19:56:50,944][44959] Updated weights for policy 1, policy_version 290 (0.0007) [2023-10-12 19:56:51,165][44958] Updated weights for policy 0, policy_version 290 (0.0007) [2023-10-12 19:56:51,339][44959] Updated weights for policy 1, policy_version 300 (0.0007) [2023-10-12 19:56:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 11178.3, 300 sec: 11178.3). Total num frames: 589824. Throughput: 0: 1589.5, 1: 1595.5. Samples: 158710. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 19:56:51,443][43579] Avg episode reward: [(0, '76.160'), (1, '79.770')] [2023-10-12 19:56:51,563][44958] Updated weights for policy 0, policy_version 300 (0.0008) [2023-10-12 19:56:51,699][44959] Updated weights for policy 1, policy_version 310 (0.0009) [2023-10-12 19:56:51,921][44958] Updated weights for policy 0, policy_version 310 (0.0009) [2023-10-12 19:56:52,050][44583] Saving new best policy, reward=79.770! [2023-10-12 19:56:52,053][44959] Updated weights for policy 1, policy_version 320 (0.0009) [2023-10-12 19:56:52,296][44518] Saving new best policy, reward=76.160! [2023-10-12 19:56:52,300][44958] Updated weights for policy 0, policy_version 320 (0.0009) [2023-10-12 19:56:56,262][44959] Updated weights for policy 1, policy_version 330 (0.0009) [2023-10-12 19:56:56,371][44958] Updated weights for policy 0, policy_version 330 (0.0008) [2023-10-12 19:56:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 11345.2, 300 sec: 11345.2). Total num frames: 655360. Throughput: 0: 1643.3, 1: 1643.6. Samples: 179044. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 19:56:56,443][43579] Avg episode reward: [(0, '78.890'), (1, '83.130')] [2023-10-12 19:56:56,631][44959] Updated weights for policy 1, policy_version 340 (0.0010) [2023-10-12 19:56:56,737][44958] Updated weights for policy 0, policy_version 340 (0.0009) [2023-10-12 19:56:56,991][44959] Updated weights for policy 1, policy_version 350 (0.0008) [2023-10-12 19:56:57,061][44583] Saving new best policy, reward=83.130! [2023-10-12 19:56:57,110][44958] Updated weights for policy 0, policy_version 350 (0.0007) [2023-10-12 19:56:57,179][44518] Saving new best policy, reward=78.890! [2023-10-12 19:57:01,063][44959] Updated weights for policy 1, policy_version 360 (0.0009) [2023-10-12 19:57:01,429][44959] Updated weights for policy 1, policy_version 370 (0.0008) [2023-10-12 19:57:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12014.9, 300 sec: 11485.6). Total num frames: 720896. Throughput: 0: 1640.5, 1: 1640.8. Samples: 199088. Policy #0 lag: (min: 26.0, avg: 39.4, max: 58.0) [2023-10-12 19:57:01,443][43579] Avg episode reward: [(0, '82.690'), (1, '81.100')] [2023-10-12 19:57:01,482][44958] Updated weights for policy 0, policy_version 360 (0.0009) [2023-10-12 19:57:01,796][44959] Updated weights for policy 1, policy_version 380 (0.0009) [2023-10-12 19:57:01,852][44958] Updated weights for policy 0, policy_version 370 (0.0008) [2023-10-12 19:57:02,219][44958] Updated weights for policy 0, policy_version 380 (0.0008) [2023-10-12 19:57:02,366][44518] Saving new best policy, reward=82.690! [2023-10-12 19:57:05,889][44959] Updated weights for policy 1, policy_version 390 (0.0009) [2023-10-12 19:57:06,257][44959] Updated weights for policy 1, policy_version 400 (0.0007) [2023-10-12 19:57:06,303][44958] Updated weights for policy 0, policy_version 390 (0.0007) [2023-10-12 19:57:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 11605.3). Total num frames: 786432. Throughput: 0: 1636.5, 1: 1647.3. Samples: 208200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 19:57:06,443][43579] Avg episode reward: [(0, '80.710'), (1, '86.930')] [2023-10-12 19:57:06,628][44959] Updated weights for policy 1, policy_version 410 (0.0008) [2023-10-12 19:57:06,665][44958] Updated weights for policy 0, policy_version 400 (0.0008) [2023-10-12 19:57:06,836][44583] Saving new best policy, reward=86.930! [2023-10-12 19:57:07,041][44958] Updated weights for policy 0, policy_version 410 (0.0009) [2023-10-12 19:57:10,669][44959] Updated weights for policy 1, policy_version 420 (0.0009) [2023-10-12 19:57:11,035][44959] Updated weights for policy 1, policy_version 430 (0.0008) [2023-10-12 19:57:11,162][44958] Updated weights for policy 0, policy_version 420 (0.0008) [2023-10-12 19:57:11,402][44959] Updated weights for policy 1, policy_version 440 (0.0008) [2023-10-12 19:57:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 11708.5). Total num frames: 851968. Throughput: 0: 1644.4, 1: 1656.8. Samples: 228682. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-12 19:57:11,443][43579] Avg episode reward: [(0, '79.210'), (1, '88.990')] [2023-10-12 19:57:11,538][44958] Updated weights for policy 0, policy_version 430 (0.0008) [2023-10-12 19:57:11,689][44583] Saving new best policy, reward=88.990! [2023-10-12 19:57:11,910][44958] Updated weights for policy 0, policy_version 440 (0.0010) [2023-10-12 19:57:15,406][44959] Updated weights for policy 1, policy_version 450 (0.0007) [2023-10-12 19:57:15,785][44959] Updated weights for policy 1, policy_version 460 (0.0009) [2023-10-12 19:57:16,070][44958] Updated weights for policy 0, policy_version 450 (0.0010) [2023-10-12 19:57:16,146][44959] Updated weights for policy 1, policy_version 470 (0.0008) [2023-10-12 19:57:16,442][44958] Updated weights for policy 0, policy_version 460 (0.0008) [2023-10-12 19:57:16,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 11798.4). Total num frames: 917504. Throughput: 0: 1645.3, 1: 1644.6. Samples: 248356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 19:57:16,443][43579] Avg episode reward: [(0, '77.040'), (1, '88.800')] [2023-10-12 19:57:16,513][44959] Updated weights for policy 1, policy_version 480 (0.0008) [2023-10-12 19:57:16,818][44958] Updated weights for policy 0, policy_version 470 (0.0009) [2023-10-12 19:57:17,183][44958] Updated weights for policy 0, policy_version 480 (0.0007) [2023-10-12 19:57:20,772][44959] Updated weights for policy 1, policy_version 490 (0.0008) [2023-10-12 19:57:21,145][44959] Updated weights for policy 1, policy_version 500 (0.0007) [2023-10-12 19:57:21,264][44958] Updated weights for policy 0, policy_version 490 (0.0008) [2023-10-12 19:57:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 11877.5). Total num frames: 983040. Throughput: 0: 1647.8, 1: 1656.2. Samples: 257878. Policy #0 lag: (min: 1.0, avg: 9.4, max: 33.0) [2023-10-12 19:57:21,443][43579] Avg episode reward: [(0, '78.670'), (1, '91.570')] [2023-10-12 19:57:21,514][44959] Updated weights for policy 1, policy_version 510 (0.0007) [2023-10-12 19:57:21,579][44583] Saving new best policy, reward=91.570! [2023-10-12 19:57:21,644][44958] Updated weights for policy 0, policy_version 500 (0.0007) [2023-10-12 19:57:22,017][44958] Updated weights for policy 0, policy_version 510 (0.0009) [2023-10-12 19:57:25,868][44959] Updated weights for policy 1, policy_version 520 (0.0007) [2023-10-12 19:57:26,119][44958] Updated weights for policy 0, policy_version 520 (0.0009) [2023-10-12 19:57:26,236][44959] Updated weights for policy 1, policy_version 530 (0.0007) [2023-10-12 19:57:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 11947.5). Total num frames: 1048576. Throughput: 0: 1656.3, 1: 1656.2. Samples: 278440. Policy #0 lag: (min: 26.0, avg: 29.7, max: 58.0) [2023-10-12 19:57:26,443][43579] Avg episode reward: [(0, '77.550'), (1, '93.070')] [2023-10-12 19:57:26,496][44958] Updated weights for policy 0, policy_version 530 (0.0009) [2023-10-12 19:57:26,597][44959] Updated weights for policy 1, policy_version 540 (0.0008) [2023-10-12 19:57:26,744][44583] Saving new best policy, reward=93.070! [2023-10-12 19:57:26,871][44958] Updated weights for policy 0, policy_version 540 (0.0008) [2023-10-12 19:57:30,707][44959] Updated weights for policy 1, policy_version 550 (0.0008) [2023-10-12 19:57:31,073][44959] Updated weights for policy 1, policy_version 560 (0.0008) [2023-10-12 19:57:31,211][44958] Updated weights for policy 0, policy_version 550 (0.0009) [2023-10-12 19:57:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12010.0). Total num frames: 1114112. Throughput: 0: 1645.4, 1: 1649.2. Samples: 297894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 19:57:31,443][44959] Updated weights for policy 1, policy_version 570 (0.0009) [2023-10-12 19:57:31,443][43579] Avg episode reward: [(0, '82.960'), (1, '90.840')] [2023-10-12 19:57:31,583][44958] Updated weights for policy 0, policy_version 560 (0.0007) [2023-10-12 19:57:31,953][44958] Updated weights for policy 0, policy_version 570 (0.0008) [2023-10-12 19:57:32,176][44518] Saving new best policy, reward=82.960! [2023-10-12 19:57:35,480][44959] Updated weights for policy 1, policy_version 580 (0.0007) [2023-10-12 19:57:35,851][44959] Updated weights for policy 1, policy_version 590 (0.0008) [2023-10-12 19:57:36,018][44958] Updated weights for policy 0, policy_version 580 (0.0009) [2023-10-12 19:57:36,214][44959] Updated weights for policy 1, policy_version 600 (0.0008) [2023-10-12 19:57:36,412][44958] Updated weights for policy 0, policy_version 590 (0.0007) [2023-10-12 19:57:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 12066.1). Total num frames: 1179648. Throughput: 0: 1646.6, 1: 1659.5. Samples: 307484. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 19:57:36,444][43579] Avg episode reward: [(0, '80.850'), (1, '92.280')] [2023-10-12 19:57:36,789][44958] Updated weights for policy 0, policy_version 600 (0.0008) [2023-10-12 19:57:40,505][44959] Updated weights for policy 1, policy_version 610 (0.0008) [2023-10-12 19:57:40,911][44959] Updated weights for policy 1, policy_version 620 (0.0007) [2023-10-12 19:57:40,954][44958] Updated weights for policy 0, policy_version 610 (0.0010) [2023-10-12 19:57:41,281][44959] Updated weights for policy 1, policy_version 630 (0.0007) [2023-10-12 19:57:41,329][44958] Updated weights for policy 0, policy_version 620 (0.0007) [2023-10-12 19:57:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12116.8). Total num frames: 1245184. Throughput: 0: 1646.4, 1: 1660.4. Samples: 327846. Policy #0 lag: (min: 17.0, avg: 18.9, max: 44.0) [2023-10-12 19:57:41,443][43579] Avg episode reward: [(0, '82.410'), (1, '92.630')] [2023-10-12 19:57:41,643][44959] Updated weights for policy 1, policy_version 640 (0.0007) [2023-10-12 19:57:41,690][44958] Updated weights for policy 0, policy_version 630 (0.0007) [2023-10-12 19:57:42,065][44958] Updated weights for policy 0, policy_version 640 (0.0010) [2023-10-12 19:57:45,640][44959] Updated weights for policy 1, policy_version 650 (0.0009) [2023-10-12 19:57:46,012][44959] Updated weights for policy 1, policy_version 660 (0.0007) [2023-10-12 19:57:46,109][44958] Updated weights for policy 0, policy_version 650 (0.0008) [2023-10-12 19:57:46,370][44959] Updated weights for policy 1, policy_version 670 (0.0008) [2023-10-12 19:57:46,449][43579] Fps is (10 sec: 16372.9, 60 sec: 13651.7, 300 sec: 12466.0). Total num frames: 1343488. Throughput: 0: 1640.7, 1: 1641.2. Samples: 346796. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 19:57:46,451][43579] Avg episode reward: [(0, '82.550'), (1, '92.190')] [2023-10-12 19:57:46,460][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000000672_688128.pth... [2023-10-12 19:57:46,479][44958] Updated weights for policy 0, policy_version 660 (0.0008) [2023-10-12 19:57:46,849][44958] Updated weights for policy 0, policy_version 670 (0.0008) [2023-10-12 19:57:46,926][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000000672_688128.pth... [2023-10-12 19:57:50,741][44959] Updated weights for policy 1, policy_version 680 (0.0008) [2023-10-12 19:57:51,102][44958] Updated weights for policy 0, policy_version 680 (0.0008) [2023-10-12 19:57:51,105][44959] Updated weights for policy 1, policy_version 690 (0.0009) [2023-10-12 19:57:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12204.6). Total num frames: 1376256. Throughput: 0: 1651.7, 1: 1652.7. Samples: 356898. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-12 19:57:51,443][43579] Avg episode reward: [(0, '83.170'), (1, '91.790')] [2023-10-12 19:57:51,473][44958] Updated weights for policy 0, policy_version 690 (0.0007) [2023-10-12 19:57:51,481][44959] Updated weights for policy 1, policy_version 700 (0.0008) [2023-10-12 19:57:51,844][44958] Updated weights for policy 0, policy_version 700 (0.0007) [2023-10-12 19:57:51,983][44518] Saving new best policy, reward=83.170! [2023-10-12 19:57:55,741][44959] Updated weights for policy 1, policy_version 710 (0.0008) [2023-10-12 19:57:56,079][44958] Updated weights for policy 0, policy_version 710 (0.0009) [2023-10-12 19:57:56,107][44959] Updated weights for policy 1, policy_version 720 (0.0008) [2023-10-12 19:57:56,443][43579] Fps is (10 sec: 9837.1, 60 sec: 13107.2, 300 sec: 12242.9). Total num frames: 1441792. Throughput: 0: 1648.9, 1: 1649.3. Samples: 377102. Policy #0 lag: (min: 15.0, avg: 16.6, max: 42.0) [2023-10-12 19:57:56,444][43579] Avg episode reward: [(0, '82.350'), (1, '91.850')] [2023-10-12 19:57:56,450][44958] Updated weights for policy 0, policy_version 720 (0.0009) [2023-10-12 19:57:56,478][44959] Updated weights for policy 1, policy_version 730 (0.0008) [2023-10-12 19:57:56,825][44958] Updated weights for policy 0, policy_version 730 (0.0010) [2023-10-12 19:58:00,552][44959] Updated weights for policy 1, policy_version 740 (0.0008) [2023-10-12 19:58:00,866][44958] Updated weights for policy 0, policy_version 740 (0.0010) [2023-10-12 19:58:00,928][44959] Updated weights for policy 1, policy_version 750 (0.0008) [2023-10-12 19:58:01,233][44958] Updated weights for policy 0, policy_version 750 (0.0009) [2023-10-12 19:58:01,286][44959] Updated weights for policy 1, policy_version 760 (0.0008) [2023-10-12 19:58:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12278.1). Total num frames: 1507328. Throughput: 0: 1642.3, 1: 1653.3. Samples: 396658. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-12 19:58:01,443][43579] Avg episode reward: [(0, '86.890'), (1, '94.860')] [2023-10-12 19:58:01,581][44583] Saving new best policy, reward=94.860! [2023-10-12 19:58:01,608][44958] Updated weights for policy 0, policy_version 760 (0.0008) [2023-10-12 19:58:01,906][44518] Saving new best policy, reward=86.890! [2023-10-12 19:58:05,297][44959] Updated weights for policy 1, policy_version 770 (0.0008) [2023-10-12 19:58:05,667][44959] Updated weights for policy 1, policy_version 780 (0.0009) [2023-10-12 19:58:05,737][44958] Updated weights for policy 0, policy_version 770 (0.0011) [2023-10-12 19:58:06,027][44959] Updated weights for policy 1, policy_version 790 (0.0008) [2023-10-12 19:58:06,109][44958] Updated weights for policy 0, policy_version 780 (0.0008) [2023-10-12 19:58:06,404][44959] Updated weights for policy 1, policy_version 800 (0.0008) [2023-10-12 19:58:06,443][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 12567.0). Total num frames: 1605632. Throughput: 0: 1649.9, 1: 1657.5. Samples: 406712. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 19:58:06,444][43579] Avg episode reward: [(0, '88.230'), (1, '95.500')] [2023-10-12 19:58:06,446][44583] Saving new best policy, reward=95.500! [2023-10-12 19:58:06,492][44958] Updated weights for policy 0, policy_version 790 (0.0008) [2023-10-12 19:58:06,856][44518] Saving new best policy, reward=88.230! [2023-10-12 19:58:06,858][44958] Updated weights for policy 0, policy_version 800 (0.0009) [2023-10-12 19:58:10,808][44959] Updated weights for policy 1, policy_version 810 (0.0011) [2023-10-12 19:58:10,978][44958] Updated weights for policy 0, policy_version 810 (0.0007) [2023-10-12 19:58:11,173][44959] Updated weights for policy 1, policy_version 820 (0.0008) [2023-10-12 19:58:11,352][44958] Updated weights for policy 0, policy_version 820 (0.0008) [2023-10-12 19:58:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12340.6). Total num frames: 1638400. Throughput: 0: 1647.6, 1: 1652.1. Samples: 426926. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 19:58:11,443][43579] Avg episode reward: [(0, '91.540'), (1, '94.400')] [2023-10-12 19:58:11,549][44959] Updated weights for policy 1, policy_version 830 (0.0008) [2023-10-12 19:58:11,726][44958] Updated weights for policy 0, policy_version 830 (0.0008) [2023-10-12 19:58:11,791][44518] Saving new best policy, reward=91.540! [2023-10-12 19:58:15,564][44959] Updated weights for policy 1, policy_version 840 (0.0008) [2023-10-12 19:58:15,938][44959] Updated weights for policy 1, policy_version 850 (0.0008) [2023-10-12 19:58:15,939][44958] Updated weights for policy 0, policy_version 840 (0.0008) [2023-10-12 19:58:16,310][44959] Updated weights for policy 1, policy_version 860 (0.0008) [2023-10-12 19:58:16,319][44958] Updated weights for policy 0, policy_version 850 (0.0008) [2023-10-12 19:58:16,442][43579] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 12368.4). Total num frames: 1703936. Throughput: 0: 1644.8, 1: 1644.1. Samples: 445892. Policy #0 lag: (min: 12.0, avg: 14.6, max: 44.0) [2023-10-12 19:58:16,443][43579] Avg episode reward: [(0, '88.940'), (1, '94.990')] [2023-10-12 19:58:16,683][44958] Updated weights for policy 0, policy_version 860 (0.0008) [2023-10-12 19:58:20,517][44959] Updated weights for policy 1, policy_version 870 (0.0009) [2023-10-12 19:58:20,855][44958] Updated weights for policy 0, policy_version 870 (0.0009) [2023-10-12 19:58:20,882][44959] Updated weights for policy 1, policy_version 880 (0.0008) [2023-10-12 19:58:21,243][44959] Updated weights for policy 1, policy_version 890 (0.0009) [2023-10-12 19:58:21,244][44958] Updated weights for policy 0, policy_version 880 (0.0009) [2023-10-12 19:58:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12394.3). Total num frames: 1769472. Throughput: 0: 1652.3, 1: 1645.9. Samples: 455900. Policy #0 lag: (min: 8.0, avg: 27.9, max: 40.0) [2023-10-12 19:58:21,444][43579] Avg episode reward: [(0, '91.470'), (1, '96.310')] [2023-10-12 19:58:21,461][44583] Saving new best policy, reward=96.310! [2023-10-12 19:58:21,605][44958] Updated weights for policy 0, policy_version 890 (0.0008) [2023-10-12 19:58:25,387][44959] Updated weights for policy 1, policy_version 900 (0.0007) [2023-10-12 19:58:25,791][44959] Updated weights for policy 1, policy_version 910 (0.0009) [2023-10-12 19:58:25,853][44958] Updated weights for policy 0, policy_version 900 (0.0009) [2023-10-12 19:58:26,157][44959] Updated weights for policy 1, policy_version 920 (0.0008) [2023-10-12 19:58:26,225][44958] Updated weights for policy 0, policy_version 910 (0.0007) [2023-10-12 19:58:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12418.4). Total num frames: 1835008. Throughput: 0: 1645.6, 1: 1650.9. Samples: 476188. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-10-12 19:58:26,443][43579] Avg episode reward: [(0, '91.460'), (1, '92.990')] [2023-10-12 19:58:26,609][44958] Updated weights for policy 0, policy_version 920 (0.0008) [2023-10-12 19:58:30,340][44959] Updated weights for policy 1, policy_version 930 (0.0008) [2023-10-12 19:58:30,706][44959] Updated weights for policy 1, policy_version 940 (0.0007) [2023-10-12 19:58:30,754][44958] Updated weights for policy 0, policy_version 930 (0.0009) [2023-10-12 19:58:31,066][44959] Updated weights for policy 1, policy_version 950 (0.0007) [2023-10-12 19:58:31,127][44958] Updated weights for policy 0, policy_version 940 (0.0007) [2023-10-12 19:58:31,436][44959] Updated weights for policy 1, policy_version 960 (0.0009) [2023-10-12 19:58:31,442][43579] Fps is (10 sec: 16384.5, 60 sec: 13653.4, 300 sec: 12655.5). Total num frames: 1933312. Throughput: 0: 1642.3, 1: 1651.1. Samples: 494974. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 19:58:31,443][43579] Avg episode reward: [(0, '90.280'), (1, '95.270')] [2023-10-12 19:58:31,499][44958] Updated weights for policy 0, policy_version 950 (0.0008) [2023-10-12 19:58:31,880][44958] Updated weights for policy 0, policy_version 960 (0.0009) [2023-10-12 19:58:35,541][44959] Updated weights for policy 1, policy_version 970 (0.0008) [2023-10-12 19:58:35,913][44959] Updated weights for policy 1, policy_version 980 (0.0007) [2023-10-12 19:58:35,966][44958] Updated weights for policy 0, policy_version 970 (0.0007) [2023-10-12 19:58:36,277][44959] Updated weights for policy 1, policy_version 990 (0.0008) [2023-10-12 19:58:36,327][44958] Updated weights for policy 0, policy_version 980 (0.0008) [2023-10-12 19:58:36,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 12669.8). Total num frames: 1998848. Throughput: 0: 1646.9, 1: 1645.0. Samples: 505032. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 19:58:36,443][43579] Avg episode reward: [(0, '88.480'), (1, '91.330')] [2023-10-12 19:58:36,694][44958] Updated weights for policy 0, policy_version 990 (0.0007) [2023-10-12 19:58:40,150][44959] Updated weights for policy 1, policy_version 1000 (0.0007) [2023-10-12 19:58:40,519][44959] Updated weights for policy 1, policy_version 1010 (0.0007) [2023-10-12 19:58:40,785][44958] Updated weights for policy 0, policy_version 1000 (0.0008) [2023-10-12 19:58:40,892][44959] Updated weights for policy 1, policy_version 1020 (0.0008) [2023-10-12 19:58:41,159][44958] Updated weights for policy 0, policy_version 1010 (0.0008) [2023-10-12 19:58:41,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 12683.2). Total num frames: 2064384. Throughput: 0: 1651.4, 1: 1647.5. Samples: 525554. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-12 19:58:41,444][43579] Avg episode reward: [(0, '86.920'), (1, '91.690')] [2023-10-12 19:58:41,531][44958] Updated weights for policy 0, policy_version 1020 (0.0007) [2023-10-12 19:58:45,079][44959] Updated weights for policy 1, policy_version 1030 (0.0009) [2023-10-12 19:58:45,447][44959] Updated weights for policy 1, policy_version 1040 (0.0007) [2023-10-12 19:58:45,723][44958] Updated weights for policy 0, policy_version 1030 (0.0008) [2023-10-12 19:58:45,820][44959] Updated weights for policy 1, policy_version 1050 (0.0008) [2023-10-12 19:58:46,099][44958] Updated weights for policy 0, policy_version 1040 (0.0008) [2023-10-12 19:58:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13108.7, 300 sec: 12695.8). Total num frames: 2129920. Throughput: 0: 1644.5, 1: 1635.2. Samples: 544248. Policy #0 lag: (min: 13.0, avg: 15.2, max: 45.0) [2023-10-12 19:58:46,443][43579] Avg episode reward: [(0, '85.310'), (1, '89.950')] [2023-10-12 19:58:46,473][44958] Updated weights for policy 0, policy_version 1050 (0.0007) [2023-10-12 19:58:49,848][44959] Updated weights for policy 1, policy_version 1060 (0.0007) [2023-10-12 19:58:50,214][44959] Updated weights for policy 1, policy_version 1070 (0.0009) [2023-10-12 19:58:50,561][44958] Updated weights for policy 0, policy_version 1060 (0.0008) [2023-10-12 19:58:50,586][44959] Updated weights for policy 1, policy_version 1080 (0.0009) [2023-10-12 19:58:50,932][44958] Updated weights for policy 0, policy_version 1070 (0.0009) [2023-10-12 19:58:51,310][44958] Updated weights for policy 0, policy_version 1080 (0.0010) [2023-10-12 19:58:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 12707.7). Total num frames: 2195456. Throughput: 0: 1644.9, 1: 1645.2. Samples: 554768. Policy #0 lag: (min: 3.0, avg: 14.3, max: 35.0) [2023-10-12 19:58:51,444][43579] Avg episode reward: [(0, '84.990'), (1, '90.070')] [2023-10-12 19:58:54,754][44959] Updated weights for policy 1, policy_version 1090 (0.0009) [2023-10-12 19:58:55,121][44959] Updated weights for policy 1, policy_version 1100 (0.0008) [2023-10-12 19:58:55,494][44959] Updated weights for policy 1, policy_version 1110 (0.0008) [2023-10-12 19:58:55,534][44958] Updated weights for policy 0, policy_version 1090 (0.0007) [2023-10-12 19:58:55,859][44959] Updated weights for policy 1, policy_version 1120 (0.0009) [2023-10-12 19:58:55,892][44958] Updated weights for policy 0, policy_version 1100 (0.0009) [2023-10-12 19:58:56,267][44958] Updated weights for policy 0, policy_version 1110 (0.0009) [2023-10-12 19:58:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 12719.0). Total num frames: 2260992. Throughput: 0: 1642.4, 1: 1644.7. Samples: 574846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 19:58:56,443][43579] Avg episode reward: [(0, '88.170'), (1, '89.420')] [2023-10-12 19:58:56,637][44958] Updated weights for policy 0, policy_version 1120 (0.0008) [2023-10-12 19:59:00,140][44959] Updated weights for policy 1, policy_version 1130 (0.0010) [2023-10-12 19:59:00,506][44959] Updated weights for policy 1, policy_version 1140 (0.0011) [2023-10-12 19:59:00,870][44958] Updated weights for policy 0, policy_version 1130 (0.0008) [2023-10-12 19:59:00,882][44959] Updated weights for policy 1, policy_version 1150 (0.0008) [2023-10-12 19:59:01,239][44958] Updated weights for policy 0, policy_version 1140 (0.0008) [2023-10-12 19:59:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 12729.6). Total num frames: 2326528. Throughput: 0: 1640.1, 1: 1641.3. Samples: 593556. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 19:59:01,443][43579] Avg episode reward: [(0, '87.360'), (1, '93.580')] [2023-10-12 19:59:01,615][44958] Updated weights for policy 0, policy_version 1150 (0.0008) [2023-10-12 19:59:04,815][44959] Updated weights for policy 1, policy_version 1160 (0.0007) [2023-10-12 19:59:05,185][44959] Updated weights for policy 1, policy_version 1170 (0.0009) [2023-10-12 19:59:05,555][44959] Updated weights for policy 1, policy_version 1180 (0.0009) [2023-10-12 19:59:05,770][44958] Updated weights for policy 0, policy_version 1160 (0.0009) [2023-10-12 19:59:06,144][44958] Updated weights for policy 0, policy_version 1170 (0.0011) [2023-10-12 19:59:06,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 12739.6). Total num frames: 2392064. Throughput: 0: 1641.6, 1: 1659.2. Samples: 604434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 19:59:06,444][43579] Avg episode reward: [(0, '88.560'), (1, '92.670')] [2023-10-12 19:59:06,522][44958] Updated weights for policy 0, policy_version 1180 (0.0010) [2023-10-12 19:59:10,045][44959] Updated weights for policy 1, policy_version 1190 (0.0009) [2023-10-12 19:59:10,422][44959] Updated weights for policy 1, policy_version 1200 (0.0008) [2023-10-12 19:59:10,689][44958] Updated weights for policy 0, policy_version 1190 (0.0009) [2023-10-12 19:59:10,781][44959] Updated weights for policy 1, policy_version 1210 (0.0008) [2023-10-12 19:59:11,060][44958] Updated weights for policy 0, policy_version 1200 (0.0008) [2023-10-12 19:59:11,427][44958] Updated weights for policy 0, policy_version 1210 (0.0008) [2023-10-12 19:59:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 12749.2). Total num frames: 2457600. Throughput: 0: 1644.3, 1: 1646.0. Samples: 624254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 19:59:11,444][43579] Avg episode reward: [(0, '89.890'), (1, '94.830')] [2023-10-12 19:59:14,956][44959] Updated weights for policy 1, policy_version 1220 (0.0009) [2023-10-12 19:59:15,346][44959] Updated weights for policy 1, policy_version 1230 (0.0008) [2023-10-12 19:59:15,553][44958] Updated weights for policy 0, policy_version 1220 (0.0008) [2023-10-12 19:59:15,717][44959] Updated weights for policy 1, policy_version 1240 (0.0007) [2023-10-12 19:59:15,916][44958] Updated weights for policy 0, policy_version 1230 (0.0007) [2023-10-12 19:59:16,283][44958] Updated weights for policy 0, policy_version 1240 (0.0009) [2023-10-12 19:59:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 12758.2). Total num frames: 2523136. Throughput: 0: 1640.1, 1: 1645.9. Samples: 642846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 19:59:16,444][43579] Avg episode reward: [(0, '92.820'), (1, '95.370')] [2023-10-12 19:59:16,578][44518] Saving new best policy, reward=92.820! [2023-10-12 19:59:19,603][44959] Updated weights for policy 1, policy_version 1250 (0.0008) [2023-10-12 19:59:19,971][44959] Updated weights for policy 1, policy_version 1260 (0.0009) [2023-10-12 19:59:20,257][44958] Updated weights for policy 0, policy_version 1250 (0.0008) [2023-10-12 19:59:20,342][44959] Updated weights for policy 1, policy_version 1270 (0.0009) [2023-10-12 19:59:20,640][44958] Updated weights for policy 0, policy_version 1260 (0.0007) [2023-10-12 19:59:20,718][44959] Updated weights for policy 1, policy_version 1280 (0.0008) [2023-10-12 19:59:21,002][44958] Updated weights for policy 0, policy_version 1270 (0.0007) [2023-10-12 19:59:21,383][44958] Updated weights for policy 0, policy_version 1280 (0.0009) [2023-10-12 19:59:21,443][43579] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 12928.4). Total num frames: 2621440. Throughput: 0: 1644.2, 1: 1662.5. Samples: 653836. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 19:59:21,444][43579] Avg episode reward: [(0, '89.530'), (1, '92.380')] [2023-10-12 19:59:25,065][44959] Updated weights for policy 1, policy_version 1290 (0.0010) [2023-10-12 19:59:25,432][44959] Updated weights for policy 1, policy_version 1300 (0.0009) [2023-10-12 19:59:25,809][44959] Updated weights for policy 1, policy_version 1310 (0.0008) [2023-10-12 19:59:25,811][44958] Updated weights for policy 0, policy_version 1290 (0.0008) [2023-10-12 19:59:26,193][44958] Updated weights for policy 0, policy_version 1300 (0.0011) [2023-10-12 19:59:26,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 12775.0). Total num frames: 2654208. Throughput: 0: 1640.2, 1: 1647.2. Samples: 673484. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 19:59:26,443][43579] Avg episode reward: [(0, '94.680'), (1, '95.740')] [2023-10-12 19:59:26,557][44958] Updated weights for policy 0, policy_version 1310 (0.0009) [2023-10-12 19:59:26,626][44518] Saving new best policy, reward=94.680! [2023-10-12 19:59:29,638][44959] Updated weights for policy 1, policy_version 1320 (0.0007) [2023-10-12 19:59:30,013][44959] Updated weights for policy 1, policy_version 1330 (0.0010) [2023-10-12 19:59:30,385][44959] Updated weights for policy 1, policy_version 1340 (0.0009) [2023-10-12 19:59:30,742][44958] Updated weights for policy 0, policy_version 1320 (0.0008) [2023-10-12 19:59:31,113][44958] Updated weights for policy 0, policy_version 1330 (0.0009) [2023-10-12 19:59:31,442][43579] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 12782.8). Total num frames: 2719744. Throughput: 0: 1637.5, 1: 1651.7. Samples: 692262. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-12 19:59:31,443][43579] Avg episode reward: [(0, '96.550'), (1, '92.870')] [2023-10-12 19:59:31,495][44958] Updated weights for policy 0, policy_version 1340 (0.0009) [2023-10-12 19:59:31,635][44518] Saving new best policy, reward=96.550! [2023-10-12 19:59:34,792][44959] Updated weights for policy 1, policy_version 1350 (0.0007) [2023-10-12 19:59:35,167][44959] Updated weights for policy 1, policy_version 1360 (0.0007) [2023-10-12 19:59:35,532][44959] Updated weights for policy 1, policy_version 1370 (0.0007) [2023-10-12 19:59:35,581][44958] Updated weights for policy 0, policy_version 1350 (0.0008) [2023-10-12 19:59:35,956][44958] Updated weights for policy 0, policy_version 1360 (0.0011) [2023-10-12 19:59:36,319][44958] Updated weights for policy 0, policy_version 1370 (0.0010) [2023-10-12 19:59:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12790.3). Total num frames: 2785280. Throughput: 0: 1646.3, 1: 1648.5. Samples: 703036. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 19:59:36,443][43579] Avg episode reward: [(0, '99.300'), (1, '90.460')] [2023-10-12 19:59:36,545][44518] Saving new best policy, reward=99.300! [2023-10-12 19:59:39,993][44959] Updated weights for policy 1, policy_version 1380 (0.0007) [2023-10-12 19:59:40,361][44959] Updated weights for policy 1, policy_version 1390 (0.0008) [2023-10-12 19:59:40,573][44958] Updated weights for policy 0, policy_version 1380 (0.0008) [2023-10-12 19:59:40,729][44959] Updated weights for policy 1, policy_version 1400 (0.0009) [2023-10-12 19:59:40,952][44958] Updated weights for policy 0, policy_version 1390 (0.0009) [2023-10-12 19:59:41,318][44958] Updated weights for policy 0, policy_version 1400 (0.0009) [2023-10-12 19:59:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12797.4). Total num frames: 2850816. Throughput: 0: 1648.9, 1: 1647.7. Samples: 723196. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-12 19:59:41,443][43579] Avg episode reward: [(0, '98.470'), (1, '95.280')] [2023-10-12 19:59:44,879][44959] Updated weights for policy 1, policy_version 1410 (0.0008) [2023-10-12 19:59:45,245][44959] Updated weights for policy 1, policy_version 1420 (0.0007) [2023-10-12 19:59:45,416][44958] Updated weights for policy 0, policy_version 1410 (0.0008) [2023-10-12 19:59:45,617][44959] Updated weights for policy 1, policy_version 1430 (0.0007) [2023-10-12 19:59:45,783][44958] Updated weights for policy 0, policy_version 1420 (0.0009) [2023-10-12 19:59:45,996][44959] Updated weights for policy 1, policy_version 1440 (0.0008) [2023-10-12 19:59:46,151][44958] Updated weights for policy 0, policy_version 1430 (0.0009) [2023-10-12 19:59:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12804.2). Total num frames: 2916352. Throughput: 0: 1644.4, 1: 1645.5. Samples: 741600. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) [2023-10-12 19:59:46,443][43579] Avg episode reward: [(0, '98.040'), (1, '99.380')] [2023-10-12 19:59:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000001440_1474560.pth... [2023-10-12 19:59:46,481][44583] Saving new best policy, reward=99.380! [2023-10-12 19:59:46,511][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000001440_1474560.pth... [2023-10-12 19:59:46,511][44958] Updated weights for policy 0, policy_version 1440 (0.0009) [2023-10-12 19:59:50,352][44959] Updated weights for policy 1, policy_version 1450 (0.0007) [2023-10-12 19:59:50,627][44958] Updated weights for policy 0, policy_version 1450 (0.0008) [2023-10-12 19:59:50,726][44959] Updated weights for policy 1, policy_version 1460 (0.0008) [2023-10-12 19:59:51,001][44958] Updated weights for policy 0, policy_version 1460 (0.0007) [2023-10-12 19:59:51,085][44959] Updated weights for policy 1, policy_version 1470 (0.0008) [2023-10-12 19:59:51,376][44958] Updated weights for policy 0, policy_version 1470 (0.0009) [2023-10-12 19:59:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12810.7). Total num frames: 2981888. Throughput: 0: 1648.3, 1: 1634.1. Samples: 752140. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 19:59:51,443][43579] Avg episode reward: [(0, '97.900'), (1, '99.330')] [2023-10-12 19:59:55,263][44959] Updated weights for policy 1, policy_version 1480 (0.0008) [2023-10-12 19:59:55,483][44958] Updated weights for policy 0, policy_version 1480 (0.0008) [2023-10-12 19:59:55,633][44959] Updated weights for policy 1, policy_version 1490 (0.0008) [2023-10-12 19:59:55,850][44958] Updated weights for policy 0, policy_version 1490 (0.0007) [2023-10-12 19:59:55,998][44959] Updated weights for policy 1, policy_version 1500 (0.0007) [2023-10-12 19:59:56,222][44958] Updated weights for policy 0, policy_version 1500 (0.0008) [2023-10-12 19:59:56,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 12954.8). Total num frames: 3080192. Throughput: 0: 1648.4, 1: 1644.0. Samples: 772412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 19:59:56,443][43579] Avg episode reward: [(0, '96.840'), (1, '98.610')] [2023-10-12 20:00:00,016][44959] Updated weights for policy 1, policy_version 1510 (0.0008) [2023-10-12 20:00:00,271][44958] Updated weights for policy 0, policy_version 1510 (0.0008) [2023-10-12 20:00:00,383][44959] Updated weights for policy 1, policy_version 1520 (0.0008) [2023-10-12 20:00:00,646][44958] Updated weights for policy 0, policy_version 1520 (0.0007) [2023-10-12 20:00:00,743][44959] Updated weights for policy 1, policy_version 1530 (0.0009) [2023-10-12 20:00:01,019][44958] Updated weights for policy 0, policy_version 1530 (0.0009) [2023-10-12 20:00:01,443][43579] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 12957.9). Total num frames: 3145728. Throughput: 0: 1639.3, 1: 1637.9. Samples: 790320. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 20:00:01,444][43579] Avg episode reward: [(0, '99.530'), (1, '100.240')] [2023-10-12 20:00:01,455][44518] Saving new best policy, reward=99.530! [2023-10-12 20:00:01,455][44583] Saving new best policy, reward=100.240! [2023-10-12 20:00:05,055][44959] Updated weights for policy 1, policy_version 1540 (0.0008) [2023-10-12 20:00:05,426][44959] Updated weights for policy 1, policy_version 1550 (0.0007) [2023-10-12 20:00:05,429][44958] Updated weights for policy 0, policy_version 1540 (0.0007) [2023-10-12 20:00:05,795][44958] Updated weights for policy 0, policy_version 1550 (0.0007) [2023-10-12 20:00:05,806][44959] Updated weights for policy 1, policy_version 1560 (0.0007) [2023-10-12 20:00:06,162][44958] Updated weights for policy 0, policy_version 1560 (0.0008) [2023-10-12 20:00:06,442][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.3, 300 sec: 12828.7). Total num frames: 3178496. Throughput: 0: 1643.4, 1: 1631.6. Samples: 801212. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 20:00:06,443][43579] Avg episode reward: [(0, '99.380'), (1, '102.920')] [2023-10-12 20:00:06,444][44583] Saving new best policy, reward=102.920! [2023-10-12 20:00:09,897][44959] Updated weights for policy 1, policy_version 1570 (0.0009) [2023-10-12 20:00:10,079][44958] Updated weights for policy 0, policy_version 1570 (0.0010) [2023-10-12 20:00:10,268][44959] Updated weights for policy 1, policy_version 1580 (0.0009) [2023-10-12 20:00:10,451][44958] Updated weights for policy 0, policy_version 1580 (0.0009) [2023-10-12 20:00:10,636][44959] Updated weights for policy 1, policy_version 1590 (0.0010) [2023-10-12 20:00:10,821][44958] Updated weights for policy 0, policy_version 1590 (0.0009) [2023-10-12 20:00:11,005][44959] Updated weights for policy 1, policy_version 1600 (0.0009) [2023-10-12 20:00:11,202][44958] Updated weights for policy 0, policy_version 1600 (0.0008) [2023-10-12 20:00:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 12963.8). Total num frames: 3276800. Throughput: 0: 1646.6, 1: 1635.6. Samples: 821182. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 20:00:11,444][43579] Avg episode reward: [(0, '107.360'), (1, '103.020')] [2023-10-12 20:00:11,445][44518] Saving new best policy, reward=107.360! [2023-10-12 20:00:11,445][44583] Saving new best policy, reward=103.020! [2023-10-12 20:00:15,424][44959] Updated weights for policy 1, policy_version 1610 (0.0007) [2023-10-12 20:00:15,757][44958] Updated weights for policy 0, policy_version 1610 (0.0007) [2023-10-12 20:00:15,803][44959] Updated weights for policy 1, policy_version 1620 (0.0009) [2023-10-12 20:00:16,132][44958] Updated weights for policy 0, policy_version 1620 (0.0007) [2023-10-12 20:00:16,170][44959] Updated weights for policy 1, policy_version 1630 (0.0008) [2023-10-12 20:00:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 12839.5). Total num frames: 3309568. Throughput: 0: 1641.3, 1: 1631.4. Samples: 839536. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 20:00:16,443][43579] Avg episode reward: [(0, '107.580'), (1, '101.570')] [2023-10-12 20:00:16,506][44958] Updated weights for policy 0, policy_version 1630 (0.0009) [2023-10-12 20:00:16,574][44518] Saving new best policy, reward=107.580! [2023-10-12 20:00:20,235][44959] Updated weights for policy 1, policy_version 1640 (0.0008) [2023-10-12 20:00:20,599][44958] Updated weights for policy 0, policy_version 1640 (0.0007) [2023-10-12 20:00:20,601][44959] Updated weights for policy 1, policy_version 1650 (0.0010) [2023-10-12 20:00:20,969][44958] Updated weights for policy 0, policy_version 1650 (0.0008) [2023-10-12 20:00:20,971][44959] Updated weights for policy 1, policy_version 1660 (0.0008) [2023-10-12 20:00:21,340][44958] Updated weights for policy 0, policy_version 1660 (0.0007) [2023-10-12 20:00:21,443][43579] Fps is (10 sec: 9830.5, 60 sec: 12561.1, 300 sec: 12844.6). Total num frames: 3375104. Throughput: 0: 1642.5, 1: 1628.7. Samples: 850240. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-12 20:00:21,443][43579] Avg episode reward: [(0, '107.820'), (1, '106.000')] [2023-10-12 20:00:21,444][44583] Saving new best policy, reward=106.000! [2023-10-12 20:00:21,483][44518] Saving new best policy, reward=107.820! [2023-10-12 20:00:25,084][44959] Updated weights for policy 1, policy_version 1670 (0.0008) [2023-10-12 20:00:25,275][44958] Updated weights for policy 0, policy_version 1670 (0.0007) [2023-10-12 20:00:25,452][44959] Updated weights for policy 1, policy_version 1680 (0.0008) [2023-10-12 20:00:25,641][44958] Updated weights for policy 0, policy_version 1680 (0.0009) [2023-10-12 20:00:25,827][44959] Updated weights for policy 1, policy_version 1690 (0.0008) [2023-10-12 20:00:26,012][44958] Updated weights for policy 0, policy_version 1690 (0.0008) [2023-10-12 20:00:26,443][43579] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 12971.8). Total num frames: 3473408. Throughput: 0: 1641.2, 1: 1630.5. Samples: 870424. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-12 20:00:26,444][43579] Avg episode reward: [(0, '107.730'), (1, '106.840')] [2023-10-12 20:00:26,445][44583] Saving new best policy, reward=106.840! [2023-10-12 20:00:29,994][44959] Updated weights for policy 1, policy_version 1700 (0.0007) [2023-10-12 20:00:30,365][44959] Updated weights for policy 1, policy_version 1710 (0.0008) [2023-10-12 20:00:30,402][44958] Updated weights for policy 0, policy_version 1700 (0.0008) [2023-10-12 20:00:30,739][44959] Updated weights for policy 1, policy_version 1720 (0.0009) [2023-10-12 20:00:30,775][44958] Updated weights for policy 0, policy_version 1710 (0.0009) [2023-10-12 20:00:31,133][44958] Updated weights for policy 0, policy_version 1720 (0.0008) [2023-10-12 20:00:31,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 12974.3). Total num frames: 3538944. Throughput: 0: 1636.6, 1: 1632.1. Samples: 888692. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-12 20:00:31,443][43579] Avg episode reward: [(0, '106.900'), (1, '105.430')] [2023-10-12 20:00:34,942][44959] Updated weights for policy 1, policy_version 1730 (0.0008) [2023-10-12 20:00:35,142][44958] Updated weights for policy 0, policy_version 1730 (0.0008) [2023-10-12 20:00:35,307][44959] Updated weights for policy 1, policy_version 1740 (0.0007) [2023-10-12 20:00:35,506][44958] Updated weights for policy 0, policy_version 1740 (0.0009) [2023-10-12 20:00:35,671][44959] Updated weights for policy 1, policy_version 1750 (0.0007) [2023-10-12 20:00:35,883][44958] Updated weights for policy 0, policy_version 1750 (0.0008) [2023-10-12 20:00:36,042][44959] Updated weights for policy 1, policy_version 1760 (0.0007) [2023-10-12 20:00:36,248][44958] Updated weights for policy 0, policy_version 1760 (0.0009) [2023-10-12 20:00:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 12976.7). Total num frames: 3604480. Throughput: 0: 1644.4, 1: 1640.1. Samples: 899940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:00:36,443][43579] Avg episode reward: [(0, '99.170'), (1, '108.950')] [2023-10-12 20:00:36,444][44583] Saving new best policy, reward=108.950! [2023-10-12 20:00:40,207][44959] Updated weights for policy 1, policy_version 1770 (0.0009) [2023-10-12 20:00:40,551][44958] Updated weights for policy 0, policy_version 1770 (0.0009) [2023-10-12 20:00:40,566][44959] Updated weights for policy 1, policy_version 1780 (0.0009) [2023-10-12 20:00:40,925][44958] Updated weights for policy 0, policy_version 1780 (0.0008) [2023-10-12 20:00:40,933][44959] Updated weights for policy 1, policy_version 1790 (0.0008) [2023-10-12 20:00:41,304][44958] Updated weights for policy 0, policy_version 1790 (0.0010) [2023-10-12 20:00:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 12979.0). Total num frames: 3670016. Throughput: 0: 1644.5, 1: 1631.1. Samples: 919816. Policy #0 lag: (min: 8.0, avg: 29.5, max: 40.0) [2023-10-12 20:00:41,444][43579] Avg episode reward: [(0, '102.910'), (1, '111.680')] [2023-10-12 20:00:41,445][44583] Saving new best policy, reward=111.680! [2023-10-12 20:00:45,211][44959] Updated weights for policy 1, policy_version 1800 (0.0007) [2023-10-12 20:00:45,348][44958] Updated weights for policy 0, policy_version 1800 (0.0009) [2023-10-12 20:00:45,591][44959] Updated weights for policy 1, policy_version 1810 (0.0007) [2023-10-12 20:00:45,720][44958] Updated weights for policy 0, policy_version 1810 (0.0007) [2023-10-12 20:00:45,957][44959] Updated weights for policy 1, policy_version 1820 (0.0008) [2023-10-12 20:00:46,089][44958] Updated weights for policy 0, policy_version 1820 (0.0007) [2023-10-12 20:00:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 12981.2). Total num frames: 3735552. Throughput: 0: 1645.6, 1: 1638.1. Samples: 938086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:00:46,443][43579] Avg episode reward: [(0, '105.390'), (1, '113.440')] [2023-10-12 20:00:46,451][44583] Saving new best policy, reward=113.440! [2023-10-12 20:00:50,095][44959] Updated weights for policy 1, policy_version 1830 (0.0008) [2023-10-12 20:00:50,181][44958] Updated weights for policy 0, policy_version 1830 (0.0007) [2023-10-12 20:00:50,459][44959] Updated weights for policy 1, policy_version 1840 (0.0009) [2023-10-12 20:00:50,548][44958] Updated weights for policy 0, policy_version 1840 (0.0008) [2023-10-12 20:00:50,831][44959] Updated weights for policy 1, policy_version 1850 (0.0008) [2023-10-12 20:00:50,923][44958] Updated weights for policy 0, policy_version 1850 (0.0008) [2023-10-12 20:00:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 12983.4). Total num frames: 3801088. Throughput: 0: 1651.7, 1: 1639.4. Samples: 949310. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 20:00:51,443][43579] Avg episode reward: [(0, '106.330'), (1, '114.140')] [2023-10-12 20:00:51,444][44583] Saving new best policy, reward=114.140! [2023-10-12 20:00:55,001][44959] Updated weights for policy 1, policy_version 1860 (0.0008) [2023-10-12 20:00:55,096][44958] Updated weights for policy 0, policy_version 1860 (0.0007) [2023-10-12 20:00:55,370][44959] Updated weights for policy 1, policy_version 1870 (0.0007) [2023-10-12 20:00:55,474][44958] Updated weights for policy 0, policy_version 1870 (0.0007) [2023-10-12 20:00:55,739][44959] Updated weights for policy 1, policy_version 1880 (0.0008) [2023-10-12 20:00:55,840][44958] Updated weights for policy 0, policy_version 1880 (0.0009) [2023-10-12 20:00:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 3866624. Throughput: 0: 1643.3, 1: 1644.8. Samples: 969148. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-12 20:00:56,443][43579] Avg episode reward: [(0, '108.500'), (1, '110.160')] [2023-10-12 20:00:56,444][44518] Saving new best policy, reward=108.500! [2023-10-12 20:00:59,743][44959] Updated weights for policy 1, policy_version 1890 (0.0007) [2023-10-12 20:00:59,777][44958] Updated weights for policy 0, policy_version 1890 (0.0010) [2023-10-12 20:01:00,112][44959] Updated weights for policy 1, policy_version 1900 (0.0009) [2023-10-12 20:01:00,149][44958] Updated weights for policy 0, policy_version 1900 (0.0008) [2023-10-12 20:01:00,473][44959] Updated weights for policy 1, policy_version 1910 (0.0009) [2023-10-12 20:01:00,529][44958] Updated weights for policy 0, policy_version 1910 (0.0010) [2023-10-12 20:01:00,844][44959] Updated weights for policy 1, policy_version 1920 (0.0007) [2023-10-12 20:01:00,895][44958] Updated weights for policy 0, policy_version 1920 (0.0011) [2023-10-12 20:01:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 3932160. Throughput: 0: 1645.1, 1: 1646.5. Samples: 987658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:01:01,443][43579] Avg episode reward: [(0, '112.030'), (1, '110.130')] [2023-10-12 20:01:01,450][44518] Saving new best policy, reward=112.030! [2023-10-12 20:01:04,938][44959] Updated weights for policy 1, policy_version 1930 (0.0008) [2023-10-12 20:01:05,125][44958] Updated weights for policy 0, policy_version 1930 (0.0008) [2023-10-12 20:01:05,296][44959] Updated weights for policy 1, policy_version 1940 (0.0009) [2023-10-12 20:01:05,493][44958] Updated weights for policy 0, policy_version 1940 (0.0007) [2023-10-12 20:01:05,670][44959] Updated weights for policy 1, policy_version 1950 (0.0008) [2023-10-12 20:01:05,870][44958] Updated weights for policy 0, policy_version 1950 (0.0007) [2023-10-12 20:01:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 3997696. Throughput: 0: 1654.6, 1: 1653.6. Samples: 999110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:01:06,443][43579] Avg episode reward: [(0, '112.250'), (1, '107.050')] [2023-10-12 20:01:06,444][44518] Saving new best policy, reward=112.250! [2023-10-12 20:01:09,895][44959] Updated weights for policy 1, policy_version 1960 (0.0008) [2023-10-12 20:01:10,118][44958] Updated weights for policy 0, policy_version 1960 (0.0008) [2023-10-12 20:01:10,264][44959] Updated weights for policy 1, policy_version 1970 (0.0010) [2023-10-12 20:01:10,481][44958] Updated weights for policy 0, policy_version 1970 (0.0008) [2023-10-12 20:01:10,633][44959] Updated weights for policy 1, policy_version 1980 (0.0008) [2023-10-12 20:01:10,855][44958] Updated weights for policy 0, policy_version 1980 (0.0010) [2023-10-12 20:01:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4063232. Throughput: 0: 1644.9, 1: 1646.2. Samples: 1018522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:01:11,443][43579] Avg episode reward: [(0, '109.940'), (1, '105.420')] [2023-10-12 20:01:14,617][44959] Updated weights for policy 1, policy_version 1990 (0.0008) [2023-10-12 20:01:14,984][44959] Updated weights for policy 1, policy_version 2000 (0.0009) [2023-10-12 20:01:15,029][44958] Updated weights for policy 0, policy_version 1990 (0.0009) [2023-10-12 20:01:15,352][44959] Updated weights for policy 1, policy_version 2010 (0.0008) [2023-10-12 20:01:15,401][44958] Updated weights for policy 0, policy_version 2000 (0.0007) [2023-10-12 20:01:15,778][44958] Updated weights for policy 0, policy_version 2010 (0.0009) [2023-10-12 20:01:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 4128768. Throughput: 0: 1646.1, 1: 1652.4. Samples: 1037126. Policy #0 lag: (min: 1.0, avg: 3.4, max: 33.0) [2023-10-12 20:01:16,443][43579] Avg episode reward: [(0, '109.850'), (1, '105.420')] [2023-10-12 20:01:19,784][44959] Updated weights for policy 1, policy_version 2020 (0.0007) [2023-10-12 20:01:19,881][44958] Updated weights for policy 0, policy_version 2020 (0.0007) [2023-10-12 20:01:20,154][44959] Updated weights for policy 1, policy_version 2030 (0.0008) [2023-10-12 20:01:20,255][44958] Updated weights for policy 0, policy_version 2030 (0.0007) [2023-10-12 20:01:20,523][44959] Updated weights for policy 1, policy_version 2040 (0.0007) [2023-10-12 20:01:20,626][44958] Updated weights for policy 0, policy_version 2040 (0.0007) [2023-10-12 20:01:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 4194304. Throughput: 0: 1652.9, 1: 1649.8. Samples: 1048562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:01:21,443][43579] Avg episode reward: [(0, '112.760'), (1, '113.140')] [2023-10-12 20:01:21,444][44518] Saving new best policy, reward=112.760! [2023-10-12 20:01:24,697][44959] Updated weights for policy 1, policy_version 2050 (0.0008) [2023-10-12 20:01:24,788][44958] Updated weights for policy 0, policy_version 2050 (0.0007) [2023-10-12 20:01:25,060][44959] Updated weights for policy 1, policy_version 2060 (0.0007) [2023-10-12 20:01:25,199][44958] Updated weights for policy 0, policy_version 2060 (0.0007) [2023-10-12 20:01:25,423][44959] Updated weights for policy 1, policy_version 2070 (0.0007) [2023-10-12 20:01:25,570][44958] Updated weights for policy 0, policy_version 2070 (0.0007) [2023-10-12 20:01:25,793][44959] Updated weights for policy 1, policy_version 2080 (0.0007) [2023-10-12 20:01:25,934][44958] Updated weights for policy 0, policy_version 2080 (0.0009) [2023-10-12 20:01:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4259840. Throughput: 0: 1640.8, 1: 1647.3. Samples: 1067784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:01:26,443][43579] Avg episode reward: [(0, '113.670'), (1, '113.780')] [2023-10-12 20:01:26,444][44518] Saving new best policy, reward=113.670! [2023-10-12 20:01:29,932][44959] Updated weights for policy 1, policy_version 2090 (0.0009) [2023-10-12 20:01:30,159][44958] Updated weights for policy 0, policy_version 2090 (0.0007) [2023-10-12 20:01:30,306][44959] Updated weights for policy 1, policy_version 2100 (0.0008) [2023-10-12 20:01:30,531][44958] Updated weights for policy 0, policy_version 2100 (0.0010) [2023-10-12 20:01:30,672][44959] Updated weights for policy 1, policy_version 2110 (0.0007) [2023-10-12 20:01:30,908][44958] Updated weights for policy 0, policy_version 2110 (0.0009) [2023-10-12 20:01:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4325376. Throughput: 0: 1642.9, 1: 1649.4. Samples: 1086242. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-12 20:01:31,443][43579] Avg episode reward: [(0, '118.430'), (1, '115.030')] [2023-10-12 20:01:31,453][44518] Saving new best policy, reward=118.430! [2023-10-12 20:01:31,453][44583] Saving new best policy, reward=115.030! [2023-10-12 20:01:35,016][44959] Updated weights for policy 1, policy_version 2120 (0.0009) [2023-10-12 20:01:35,207][44958] Updated weights for policy 0, policy_version 2120 (0.0010) [2023-10-12 20:01:35,402][44959] Updated weights for policy 1, policy_version 2130 (0.0009) [2023-10-12 20:01:35,584][44958] Updated weights for policy 0, policy_version 2130 (0.0007) [2023-10-12 20:01:35,775][44959] Updated weights for policy 1, policy_version 2140 (0.0007) [2023-10-12 20:01:35,962][44958] Updated weights for policy 0, policy_version 2140 (0.0008) [2023-10-12 20:01:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4390912. Throughput: 0: 1639.0, 1: 1648.3. Samples: 1097236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:01:36,443][43579] Avg episode reward: [(0, '121.660'), (1, '121.260')] [2023-10-12 20:01:36,444][44518] Saving new best policy, reward=121.660! [2023-10-12 20:01:36,444][44583] Saving new best policy, reward=121.260! [2023-10-12 20:01:39,720][44959] Updated weights for policy 1, policy_version 2150 (0.0007) [2023-10-12 20:01:40,095][44959] Updated weights for policy 1, policy_version 2160 (0.0007) [2023-10-12 20:01:40,255][44958] Updated weights for policy 0, policy_version 2150 (0.0008) [2023-10-12 20:01:40,460][44959] Updated weights for policy 1, policy_version 2170 (0.0007) [2023-10-12 20:01:40,633][44958] Updated weights for policy 0, policy_version 2160 (0.0007) [2023-10-12 20:01:41,005][44958] Updated weights for policy 0, policy_version 2170 (0.0008) [2023-10-12 20:01:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4456448. Throughput: 0: 1642.9, 1: 1637.5. Samples: 1116766. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-12 20:01:41,443][43579] Avg episode reward: [(0, '124.310'), (1, '124.210')] [2023-10-12 20:01:41,444][44583] Saving new best policy, reward=124.210! [2023-10-12 20:01:41,444][44518] Saving new best policy, reward=124.310! [2023-10-12 20:01:44,603][44959] Updated weights for policy 1, policy_version 2180 (0.0009) [2023-10-12 20:01:44,973][44959] Updated weights for policy 1, policy_version 2190 (0.0007) [2023-10-12 20:01:45,157][44958] Updated weights for policy 0, policy_version 2180 (0.0008) [2023-10-12 20:01:45,339][44959] Updated weights for policy 1, policy_version 2200 (0.0007) [2023-10-12 20:01:45,533][44958] Updated weights for policy 0, policy_version 2190 (0.0007) [2023-10-12 20:01:45,903][44958] Updated weights for policy 0, policy_version 2200 (0.0008) [2023-10-12 20:01:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 4521984. Throughput: 0: 1639.7, 1: 1643.7. Samples: 1135412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:01:46,444][43579] Avg episode reward: [(0, '126.720'), (1, '124.870')] [2023-10-12 20:01:46,456][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000002208_2260992.pth... [2023-10-12 20:01:46,456][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000002208_2260992.pth... [2023-10-12 20:01:46,489][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000000672_688128.pth [2023-10-12 20:01:46,492][44518] Saving new best policy, reward=126.720! [2023-10-12 20:01:46,494][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000000672_688128.pth [2023-10-12 20:01:46,498][44583] Saving new best policy, reward=124.870! [2023-10-12 20:01:49,524][44959] Updated weights for policy 1, policy_version 2210 (0.0008) [2023-10-12 20:01:49,896][44959] Updated weights for policy 1, policy_version 2220 (0.0009) [2023-10-12 20:01:50,175][44958] Updated weights for policy 0, policy_version 2210 (0.0008) [2023-10-12 20:01:50,255][44959] Updated weights for policy 1, policy_version 2230 (0.0009) [2023-10-12 20:01:50,539][44958] Updated weights for policy 0, policy_version 2220 (0.0009) [2023-10-12 20:01:50,626][44959] Updated weights for policy 1, policy_version 2240 (0.0008) [2023-10-12 20:01:50,913][44958] Updated weights for policy 0, policy_version 2230 (0.0007) [2023-10-12 20:01:51,276][44958] Updated weights for policy 0, policy_version 2240 (0.0008) [2023-10-12 20:01:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4587520. Throughput: 0: 1632.7, 1: 1641.7. Samples: 1146458. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-10-12 20:01:51,443][43579] Avg episode reward: [(0, '126.670'), (1, '128.370')] [2023-10-12 20:01:51,444][44583] Saving new best policy, reward=128.370! [2023-10-12 20:01:54,709][44959] Updated weights for policy 1, policy_version 2250 (0.0007) [2023-10-12 20:01:55,076][44959] Updated weights for policy 1, policy_version 2260 (0.0007) [2023-10-12 20:01:55,452][44959] Updated weights for policy 1, policy_version 2270 (0.0007) [2023-10-12 20:01:55,461][44958] Updated weights for policy 0, policy_version 2250 (0.0009) [2023-10-12 20:01:55,829][44958] Updated weights for policy 0, policy_version 2260 (0.0010) [2023-10-12 20:01:56,197][44958] Updated weights for policy 0, policy_version 2270 (0.0011) [2023-10-12 20:01:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 4653056. Throughput: 0: 1641.1, 1: 1639.3. Samples: 1166142. Policy #0 lag: (min: 31.0, avg: 32.8, max: 61.0) [2023-10-12 20:01:56,444][43579] Avg episode reward: [(0, '123.910'), (1, '132.290')] [2023-10-12 20:01:56,445][44583] Saving new best policy, reward=132.290! [2023-10-12 20:01:59,716][44959] Updated weights for policy 1, policy_version 2280 (0.0007) [2023-10-12 20:02:00,089][44959] Updated weights for policy 1, policy_version 2290 (0.0008) [2023-10-12 20:02:00,339][44958] Updated weights for policy 0, policy_version 2280 (0.0010) [2023-10-12 20:02:00,463][44959] Updated weights for policy 1, policy_version 2300 (0.0007) [2023-10-12 20:02:00,712][44958] Updated weights for policy 0, policy_version 2290 (0.0008) [2023-10-12 20:02:01,090][44958] Updated weights for policy 0, policy_version 2300 (0.0007) [2023-10-12 20:02:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4718592. Throughput: 0: 1636.6, 1: 1643.6. Samples: 1184736. Policy #0 lag: (min: 9.0, avg: 14.3, max: 41.0) [2023-10-12 20:02:01,443][43579] Avg episode reward: [(0, '122.660'), (1, '132.250')] [2023-10-12 20:02:04,574][44959] Updated weights for policy 1, policy_version 2310 (0.0009) [2023-10-12 20:02:04,943][44959] Updated weights for policy 1, policy_version 2320 (0.0009) [2023-10-12 20:02:05,160][44958] Updated weights for policy 0, policy_version 2310 (0.0008) [2023-10-12 20:02:05,308][44959] Updated weights for policy 1, policy_version 2330 (0.0007) [2023-10-12 20:02:05,542][44958] Updated weights for policy 0, policy_version 2320 (0.0008) [2023-10-12 20:02:05,927][44958] Updated weights for policy 0, policy_version 2330 (0.0009) [2023-10-12 20:02:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 4784128. Throughput: 0: 1630.9, 1: 1647.6. Samples: 1196096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:02:06,444][43579] Avg episode reward: [(0, '120.660'), (1, '137.220')] [2023-10-12 20:02:06,445][44583] Saving new best policy, reward=137.220! [2023-10-12 20:02:09,494][44959] Updated weights for policy 1, policy_version 2340 (0.0007) [2023-10-12 20:02:09,859][44959] Updated weights for policy 1, policy_version 2350 (0.0008) [2023-10-12 20:02:10,210][44958] Updated weights for policy 0, policy_version 2340 (0.0009) [2023-10-12 20:02:10,236][44959] Updated weights for policy 1, policy_version 2360 (0.0010) [2023-10-12 20:02:10,601][44958] Updated weights for policy 0, policy_version 2350 (0.0008) [2023-10-12 20:02:10,973][44958] Updated weights for policy 0, policy_version 2360 (0.0009) [2023-10-12 20:02:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4849664. Throughput: 0: 1635.3, 1: 1638.4. Samples: 1215102. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 20:02:11,443][43579] Avg episode reward: [(0, '119.870'), (1, '137.140')] [2023-10-12 20:02:14,375][44959] Updated weights for policy 1, policy_version 2370 (0.0009) [2023-10-12 20:02:14,741][44959] Updated weights for policy 1, policy_version 2380 (0.0009) [2023-10-12 20:02:15,113][44959] Updated weights for policy 1, policy_version 2390 (0.0008) [2023-10-12 20:02:15,129][44958] Updated weights for policy 0, policy_version 2370 (0.0009) [2023-10-12 20:02:15,475][44959] Updated weights for policy 1, policy_version 2400 (0.0009) [2023-10-12 20:02:15,492][44958] Updated weights for policy 0, policy_version 2380 (0.0008) [2023-10-12 20:02:15,864][44958] Updated weights for policy 0, policy_version 2390 (0.0009) [2023-10-12 20:02:16,232][44958] Updated weights for policy 0, policy_version 2400 (0.0011) [2023-10-12 20:02:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4915200. Throughput: 0: 1628.3, 1: 1652.7. Samples: 1233884. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 20:02:16,443][43579] Avg episode reward: [(0, '120.860'), (1, '137.130')] [2023-10-12 20:02:19,691][44959] Updated weights for policy 1, policy_version 2410 (0.0007) [2023-10-12 20:02:20,055][44959] Updated weights for policy 1, policy_version 2420 (0.0007) [2023-10-12 20:02:20,416][44959] Updated weights for policy 1, policy_version 2430 (0.0007) [2023-10-12 20:02:20,467][44958] Updated weights for policy 0, policy_version 2410 (0.0008) [2023-10-12 20:02:20,838][44958] Updated weights for policy 0, policy_version 2420 (0.0008) [2023-10-12 20:02:21,208][44958] Updated weights for policy 0, policy_version 2430 (0.0007) [2023-10-12 20:02:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4980736. Throughput: 0: 1632.2, 1: 1656.9. Samples: 1245246. Policy #0 lag: (min: 4.0, avg: 7.3, max: 36.0) [2023-10-12 20:02:21,443][43579] Avg episode reward: [(0, '123.340'), (1, '137.560')] [2023-10-12 20:02:21,444][44583] Saving new best policy, reward=137.560! [2023-10-12 20:02:24,572][44959] Updated weights for policy 1, policy_version 2440 (0.0007) [2023-10-12 20:02:24,941][44959] Updated weights for policy 1, policy_version 2450 (0.0008) [2023-10-12 20:02:25,304][44959] Updated weights for policy 1, policy_version 2460 (0.0007) [2023-10-12 20:02:25,546][44958] Updated weights for policy 0, policy_version 2440 (0.0008) [2023-10-12 20:02:25,916][44958] Updated weights for policy 0, policy_version 2450 (0.0007) [2023-10-12 20:02:26,290][44958] Updated weights for policy 0, policy_version 2460 (0.0009) [2023-10-12 20:02:26,447][43579] Fps is (10 sec: 13101.7, 60 sec: 13106.3, 300 sec: 13329.2). Total num frames: 5046272. Throughput: 0: 1630.1, 1: 1654.6. Samples: 1264590. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-12 20:02:26,448][43579] Avg episode reward: [(0, '125.420'), (1, '138.130')] [2023-10-12 20:02:26,449][44583] Saving new best policy, reward=138.130! [2023-10-12 20:02:29,563][44959] Updated weights for policy 1, policy_version 2470 (0.0007) [2023-10-12 20:02:29,928][44959] Updated weights for policy 1, policy_version 2480 (0.0009) [2023-10-12 20:02:30,298][44959] Updated weights for policy 1, policy_version 2490 (0.0010) [2023-10-12 20:02:30,464][44958] Updated weights for policy 0, policy_version 2470 (0.0008) [2023-10-12 20:02:30,836][44958] Updated weights for policy 0, policy_version 2480 (0.0009) [2023-10-12 20:02:31,214][44958] Updated weights for policy 0, policy_version 2490 (0.0007) [2023-10-12 20:02:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5111808. Throughput: 0: 1632.2, 1: 1654.4. Samples: 1283308. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 20:02:31,443][43579] Avg episode reward: [(0, '126.480'), (1, '138.170')] [2023-10-12 20:02:31,452][44583] Saving new best policy, reward=138.170! [2023-10-12 20:02:34,369][44959] Updated weights for policy 1, policy_version 2500 (0.0009) [2023-10-12 20:02:34,731][44959] Updated weights for policy 1, policy_version 2510 (0.0007) [2023-10-12 20:02:35,104][44959] Updated weights for policy 1, policy_version 2520 (0.0009) [2023-10-12 20:02:35,404][44958] Updated weights for policy 0, policy_version 2500 (0.0008) [2023-10-12 20:02:35,784][44958] Updated weights for policy 0, policy_version 2510 (0.0008) [2023-10-12 20:02:36,167][44958] Updated weights for policy 0, policy_version 2520 (0.0007) [2023-10-12 20:02:36,442][43579] Fps is (10 sec: 9834.5, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 5144576. Throughput: 0: 1623.9, 1: 1652.4. Samples: 1293894. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 20:02:36,443][43579] Avg episode reward: [(0, '129.780'), (1, '136.550')] [2023-10-12 20:02:36,459][44518] Saving new best policy, reward=129.780! [2023-10-12 20:02:39,369][44959] Updated weights for policy 1, policy_version 2530 (0.0008) [2023-10-12 20:02:39,746][44959] Updated weights for policy 1, policy_version 2540 (0.0008) [2023-10-12 20:02:40,115][44959] Updated weights for policy 1, policy_version 2550 (0.0009) [2023-10-12 20:02:40,306][44958] Updated weights for policy 0, policy_version 2530 (0.0009) [2023-10-12 20:02:40,482][44959] Updated weights for policy 1, policy_version 2560 (0.0009) [2023-10-12 20:02:40,684][44958] Updated weights for policy 0, policy_version 2540 (0.0009) [2023-10-12 20:02:41,061][44958] Updated weights for policy 0, policy_version 2550 (0.0008) [2023-10-12 20:02:41,436][44958] Updated weights for policy 0, policy_version 2560 (0.0008) [2023-10-12 20:02:41,444][43579] Fps is (10 sec: 13104.9, 60 sec: 13106.8, 300 sec: 13218.5). Total num frames: 5242880. Throughput: 0: 1622.8, 1: 1648.6. Samples: 1313358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:02:41,446][43579] Avg episode reward: [(0, '132.440'), (1, '139.290')] [2023-10-12 20:02:41,447][44518] Saving new best policy, reward=132.440! [2023-10-12 20:02:41,447][44583] Saving new best policy, reward=139.290! [2023-10-12 20:02:44,504][44959] Updated weights for policy 1, policy_version 2570 (0.0008) [2023-10-12 20:02:44,877][44959] Updated weights for policy 1, policy_version 2580 (0.0007) [2023-10-12 20:02:45,244][44959] Updated weights for policy 1, policy_version 2590 (0.0008) [2023-10-12 20:02:45,549][44958] Updated weights for policy 0, policy_version 2570 (0.0007) [2023-10-12 20:02:45,925][44958] Updated weights for policy 0, policy_version 2580 (0.0010) [2023-10-12 20:02:46,300][44958] Updated weights for policy 0, policy_version 2590 (0.0009) [2023-10-12 20:02:46,443][43579] Fps is (10 sec: 16383.6, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 5308416. Throughput: 0: 1627.9, 1: 1654.7. Samples: 1332454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:02:46,444][43579] Avg episode reward: [(0, '133.710'), (1, '138.200')] [2023-10-12 20:02:46,456][44518] Saving new best policy, reward=133.710! [2023-10-12 20:02:49,401][44959] Updated weights for policy 1, policy_version 2600 (0.0008) [2023-10-12 20:02:49,771][44959] Updated weights for policy 1, policy_version 2610 (0.0007) [2023-10-12 20:02:50,139][44959] Updated weights for policy 1, policy_version 2620 (0.0007) [2023-10-12 20:02:50,388][44958] Updated weights for policy 0, policy_version 2600 (0.0008) [2023-10-12 20:02:50,762][44958] Updated weights for policy 0, policy_version 2610 (0.0007) [2023-10-12 20:02:51,127][44958] Updated weights for policy 0, policy_version 2620 (0.0007) [2023-10-12 20:02:51,443][43579] Fps is (10 sec: 13109.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5373952. Throughput: 0: 1620.7, 1: 1648.0. Samples: 1343184. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 20:02:51,443][43579] Avg episode reward: [(0, '130.660'), (1, '139.300')] [2023-10-12 20:02:51,444][44583] Saving new best policy, reward=139.300! [2023-10-12 20:02:54,240][44959] Updated weights for policy 1, policy_version 2630 (0.0007) [2023-10-12 20:02:54,617][44959] Updated weights for policy 1, policy_version 2640 (0.0010) [2023-10-12 20:02:54,990][44959] Updated weights for policy 1, policy_version 2650 (0.0011) [2023-10-12 20:02:55,519][44958] Updated weights for policy 0, policy_version 2630 (0.0007) [2023-10-12 20:02:55,901][44958] Updated weights for policy 0, policy_version 2640 (0.0009) [2023-10-12 20:02:56,270][44958] Updated weights for policy 0, policy_version 2650 (0.0009) [2023-10-12 20:02:56,442][43579] Fps is (10 sec: 9830.7, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 5406720. Throughput: 0: 1628.1, 1: 1650.0. Samples: 1362614. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 20:02:56,443][43579] Avg episode reward: [(0, '132.150'), (1, '138.950')] [2023-10-12 20:02:59,279][44959] Updated weights for policy 1, policy_version 2660 (0.0009) [2023-10-12 20:02:59,657][44959] Updated weights for policy 1, policy_version 2670 (0.0010) [2023-10-12 20:03:00,024][44959] Updated weights for policy 1, policy_version 2680 (0.0009) [2023-10-12 20:03:00,362][44958] Updated weights for policy 0, policy_version 2660 (0.0008) [2023-10-12 20:03:00,736][44958] Updated weights for policy 0, policy_version 2670 (0.0007) [2023-10-12 20:03:01,097][44958] Updated weights for policy 0, policy_version 2680 (0.0009) [2023-10-12 20:03:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 5505024. Throughput: 0: 1641.5, 1: 1651.1. Samples: 1382052. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 20:03:01,444][43579] Avg episode reward: [(0, '132.340'), (1, '141.740')] [2023-10-12 20:03:01,453][44583] Saving new best policy, reward=141.740! [2023-10-12 20:03:04,159][44959] Updated weights for policy 1, policy_version 2690 (0.0008) [2023-10-12 20:03:04,570][44959] Updated weights for policy 1, policy_version 2700 (0.0007) [2023-10-12 20:03:04,948][44959] Updated weights for policy 1, policy_version 2710 (0.0009) [2023-10-12 20:03:05,156][44958] Updated weights for policy 0, policy_version 2690 (0.0008) [2023-10-12 20:03:05,313][44959] Updated weights for policy 1, policy_version 2720 (0.0010) [2023-10-12 20:03:05,518][44958] Updated weights for policy 0, policy_version 2700 (0.0007) [2023-10-12 20:03:05,896][44958] Updated weights for policy 0, policy_version 2710 (0.0009) [2023-10-12 20:03:06,272][44958] Updated weights for policy 0, policy_version 2720 (0.0008) [2023-10-12 20:03:06,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5570560. Throughput: 0: 1634.3, 1: 1648.9. Samples: 1392992. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 20:03:06,443][43579] Avg episode reward: [(0, '131.590'), (1, '143.790')] [2023-10-12 20:03:06,444][44583] Saving new best policy, reward=143.790! [2023-10-12 20:03:09,383][44959] Updated weights for policy 1, policy_version 2730 (0.0007) [2023-10-12 20:03:09,769][44959] Updated weights for policy 1, policy_version 2740 (0.0007) [2023-10-12 20:03:10,133][44959] Updated weights for policy 1, policy_version 2750 (0.0009) [2023-10-12 20:03:10,342][44958] Updated weights for policy 0, policy_version 2730 (0.0009) [2023-10-12 20:03:10,711][44958] Updated weights for policy 0, policy_version 2740 (0.0009) [2023-10-12 20:03:11,091][44958] Updated weights for policy 0, policy_version 2750 (0.0009) [2023-10-12 20:03:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 5636096. Throughput: 0: 1636.7, 1: 1642.6. Samples: 1412146. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 20:03:11,444][43579] Avg episode reward: [(0, '131.940'), (1, '141.450')] [2023-10-12 20:03:14,234][44959] Updated weights for policy 1, policy_version 2760 (0.0007) [2023-10-12 20:03:14,599][44959] Updated weights for policy 1, policy_version 2770 (0.0007) [2023-10-12 20:03:14,967][44959] Updated weights for policy 1, policy_version 2780 (0.0007) [2023-10-12 20:03:15,208][44958] Updated weights for policy 0, policy_version 2760 (0.0008) [2023-10-12 20:03:15,577][44958] Updated weights for policy 0, policy_version 2770 (0.0008) [2023-10-12 20:03:15,958][44958] Updated weights for policy 0, policy_version 2780 (0.0009) [2023-10-12 20:03:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5701632. Throughput: 0: 1634.0, 1: 1650.7. Samples: 1431120. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-12 20:03:16,443][43579] Avg episode reward: [(0, '132.930'), (1, '146.310')] [2023-10-12 20:03:16,453][44583] Saving new best policy, reward=146.310! [2023-10-12 20:03:18,937][44959] Updated weights for policy 1, policy_version 2790 (0.0009) [2023-10-12 20:03:19,315][44959] Updated weights for policy 1, policy_version 2800 (0.0007) [2023-10-12 20:03:19,684][44959] Updated weights for policy 1, policy_version 2810 (0.0009) [2023-10-12 20:03:20,191][44958] Updated weights for policy 0, policy_version 2790 (0.0010) [2023-10-12 20:03:20,563][44958] Updated weights for policy 0, policy_version 2800 (0.0008) [2023-10-12 20:03:20,940][44958] Updated weights for policy 0, policy_version 2810 (0.0010) [2023-10-12 20:03:21,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 5767168. Throughput: 0: 1646.4, 1: 1652.8. Samples: 1442356. Policy #0 lag: (min: 19.0, avg: 26.2, max: 51.0) [2023-10-12 20:03:21,443][43579] Avg episode reward: [(0, '129.480'), (1, '143.480')] [2023-10-12 20:03:23,757][44959] Updated weights for policy 1, policy_version 2820 (0.0007) [2023-10-12 20:03:24,129][44959] Updated weights for policy 1, policy_version 2830 (0.0007) [2023-10-12 20:03:24,499][44959] Updated weights for policy 1, policy_version 2840 (0.0007) [2023-10-12 20:03:25,175][44958] Updated weights for policy 0, policy_version 2820 (0.0007) [2023-10-12 20:03:25,545][44958] Updated weights for policy 0, policy_version 2830 (0.0009) [2023-10-12 20:03:25,909][44958] Updated weights for policy 0, policy_version 2840 (0.0008) [2023-10-12 20:03:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13108.1, 300 sec: 13218.3). Total num frames: 5832704. Throughput: 0: 1644.6, 1: 1650.9. Samples: 1461648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:03:26,443][43579] Avg episode reward: [(0, '128.460'), (1, '148.400')] [2023-10-12 20:03:26,444][44583] Saving new best policy, reward=148.400! [2023-10-12 20:03:28,655][44959] Updated weights for policy 1, policy_version 2850 (0.0008) [2023-10-12 20:03:29,023][44959] Updated weights for policy 1, policy_version 2860 (0.0008) [2023-10-12 20:03:29,390][44959] Updated weights for policy 1, policy_version 2870 (0.0007) [2023-10-12 20:03:29,762][44959] Updated weights for policy 1, policy_version 2880 (0.0008) [2023-10-12 20:03:30,187][44958] Updated weights for policy 0, policy_version 2850 (0.0009) [2023-10-12 20:03:30,560][44958] Updated weights for policy 0, policy_version 2860 (0.0009) [2023-10-12 20:03:30,922][44958] Updated weights for policy 0, policy_version 2870 (0.0008) [2023-10-12 20:03:31,301][44958] Updated weights for policy 0, policy_version 2880 (0.0009) [2023-10-12 20:03:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 5898240. Throughput: 0: 1643.7, 1: 1660.4. Samples: 1481138. Policy #0 lag: (min: 16.0, avg: 42.7, max: 48.0) [2023-10-12 20:03:31,443][43579] Avg episode reward: [(0, '127.460'), (1, '151.780')] [2023-10-12 20:03:31,450][44583] Saving new best policy, reward=151.780! [2023-10-12 20:03:33,731][44959] Updated weights for policy 1, policy_version 2890 (0.0009) [2023-10-12 20:03:34,098][44959] Updated weights for policy 1, policy_version 2900 (0.0007) [2023-10-12 20:03:34,471][44959] Updated weights for policy 1, policy_version 2910 (0.0008) [2023-10-12 20:03:35,637][44958] Updated weights for policy 0, policy_version 2890 (0.0008) [2023-10-12 20:03:36,011][44958] Updated weights for policy 0, policy_version 2900 (0.0008) [2023-10-12 20:03:36,384][44958] Updated weights for policy 0, policy_version 2910 (0.0010) [2023-10-12 20:03:36,442][43579] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 5931008. Throughput: 0: 1645.3, 1: 1652.4. Samples: 1491584. Policy #0 lag: (min: 16.0, avg: 42.7, max: 48.0) [2023-10-12 20:03:36,443][43579] Avg episode reward: [(0, '126.590'), (1, '153.370')] [2023-10-12 20:03:36,444][44583] Saving new best policy, reward=153.370! [2023-10-12 20:03:38,734][44959] Updated weights for policy 1, policy_version 2920 (0.0010) [2023-10-12 20:03:39,098][44959] Updated weights for policy 1, policy_version 2930 (0.0010) [2023-10-12 20:03:39,477][44959] Updated weights for policy 1, policy_version 2940 (0.0009) [2023-10-12 20:03:40,597][44958] Updated weights for policy 0, policy_version 2920 (0.0009) [2023-10-12 20:03:40,978][44958] Updated weights for policy 0, policy_version 2930 (0.0010) [2023-10-12 20:03:41,346][44958] Updated weights for policy 0, policy_version 2940 (0.0010) [2023-10-12 20:03:41,443][43579] Fps is (10 sec: 9830.2, 60 sec: 12561.4, 300 sec: 13107.2). Total num frames: 5996544. Throughput: 0: 1644.1, 1: 1653.1. Samples: 1510990. Policy #0 lag: (min: 15.0, avg: 17.9, max: 47.0) [2023-10-12 20:03:41,443][43579] Avg episode reward: [(0, '130.340'), (1, '150.180')] [2023-10-12 20:03:43,618][44959] Updated weights for policy 1, policy_version 2950 (0.0008) [2023-10-12 20:03:43,978][44959] Updated weights for policy 1, policy_version 2960 (0.0008) [2023-10-12 20:03:44,345][44959] Updated weights for policy 1, policy_version 2970 (0.0008) [2023-10-12 20:03:45,523][44958] Updated weights for policy 0, policy_version 2950 (0.0009) [2023-10-12 20:03:45,891][44958] Updated weights for policy 0, policy_version 2960 (0.0009) [2023-10-12 20:03:46,266][44958] Updated weights for policy 0, policy_version 2970 (0.0009) [2023-10-12 20:03:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 6062080. Throughput: 0: 1635.7, 1: 1658.6. Samples: 1530292. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-12 20:03:46,443][43579] Avg episode reward: [(0, '128.360'), (1, '155.880')] [2023-10-12 20:03:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000002976_3047424.pth... [2023-10-12 20:03:46,488][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000002976_3047424.pth... [2023-10-12 20:03:46,489][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000001440_1474560.pth [2023-10-12 20:03:46,493][44583] Saving new best policy, reward=155.880! [2023-10-12 20:03:46,529][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000001440_1474560.pth [2023-10-12 20:03:48,378][44959] Updated weights for policy 1, policy_version 2980 (0.0010) [2023-10-12 20:03:48,745][44959] Updated weights for policy 1, policy_version 2990 (0.0011) [2023-10-12 20:03:49,123][44959] Updated weights for policy 1, policy_version 3000 (0.0010) [2023-10-12 20:03:50,531][44958] Updated weights for policy 0, policy_version 2980 (0.0010) [2023-10-12 20:03:50,903][44958] Updated weights for policy 0, policy_version 2990 (0.0009) [2023-10-12 20:03:51,271][44958] Updated weights for policy 0, policy_version 3000 (0.0009) [2023-10-12 20:03:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 6127616. Throughput: 0: 1631.7, 1: 1642.3. Samples: 1540322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:03:51,443][43579] Avg episode reward: [(0, '130.210'), (1, '152.960')] [2023-10-12 20:03:53,398][44959] Updated weights for policy 1, policy_version 3010 (0.0007) [2023-10-12 20:03:53,767][44959] Updated weights for policy 1, policy_version 3020 (0.0009) [2023-10-12 20:03:54,128][44959] Updated weights for policy 1, policy_version 3030 (0.0009) [2023-10-12 20:03:54,502][44959] Updated weights for policy 1, policy_version 3040 (0.0010) [2023-10-12 20:03:55,358][44958] Updated weights for policy 0, policy_version 3010 (0.0008) [2023-10-12 20:03:55,734][44958] Updated weights for policy 0, policy_version 3020 (0.0011) [2023-10-12 20:03:56,100][44958] Updated weights for policy 0, policy_version 3030 (0.0009) [2023-10-12 20:03:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6193152. Throughput: 0: 1639.3, 1: 1653.6. Samples: 1560326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:03:56,443][43579] Avg episode reward: [(0, '130.410'), (1, '149.030')] [2023-10-12 20:03:56,474][44958] Updated weights for policy 0, policy_version 3040 (0.0007) [2023-10-12 20:03:58,640][44959] Updated weights for policy 1, policy_version 3050 (0.0008) [2023-10-12 20:03:59,016][44959] Updated weights for policy 1, policy_version 3060 (0.0008) [2023-10-12 20:03:59,391][44959] Updated weights for policy 1, policy_version 3070 (0.0009) [2023-10-12 20:04:00,773][44958] Updated weights for policy 0, policy_version 3050 (0.0010) [2023-10-12 20:04:01,145][44958] Updated weights for policy 0, policy_version 3060 (0.0009) [2023-10-12 20:04:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 6258688. Throughput: 0: 1640.4, 1: 1663.7. Samples: 1579804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:04:01,443][43579] Avg episode reward: [(0, '129.400'), (1, '148.750')] [2023-10-12 20:04:01,519][44958] Updated weights for policy 0, policy_version 3070 (0.0007) [2023-10-12 20:04:03,273][44959] Updated weights for policy 1, policy_version 3080 (0.0007) [2023-10-12 20:04:03,643][44959] Updated weights for policy 1, policy_version 3090 (0.0008) [2023-10-12 20:04:04,007][44959] Updated weights for policy 1, policy_version 3100 (0.0008) [2023-10-12 20:04:05,763][44958] Updated weights for policy 0, policy_version 3080 (0.0010) [2023-10-12 20:04:06,135][44958] Updated weights for policy 0, policy_version 3090 (0.0009) [2023-10-12 20:04:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 6324224. Throughput: 0: 1631.6, 1: 1644.4. Samples: 1589776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:04:06,443][43579] Avg episode reward: [(0, '127.820'), (1, '152.910')] [2023-10-12 20:04:06,513][44958] Updated weights for policy 0, policy_version 3100 (0.0009) [2023-10-12 20:04:08,257][44959] Updated weights for policy 1, policy_version 3110 (0.0008) [2023-10-12 20:04:08,624][44959] Updated weights for policy 1, policy_version 3120 (0.0008) [2023-10-12 20:04:09,006][44959] Updated weights for policy 1, policy_version 3130 (0.0009) [2023-10-12 20:04:10,668][44958] Updated weights for policy 0, policy_version 3110 (0.0008) [2023-10-12 20:04:11,039][44958] Updated weights for policy 0, policy_version 3120 (0.0007) [2023-10-12 20:04:11,404][44958] Updated weights for policy 0, policy_version 3130 (0.0008) [2023-10-12 20:04:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 6389760. Throughput: 0: 1636.5, 1: 1659.3. Samples: 1609958. Policy #0 lag: (min: 3.0, avg: 4.2, max: 27.0) [2023-10-12 20:04:11,443][43579] Avg episode reward: [(0, '128.970'), (1, '156.280')] [2023-10-12 20:04:11,444][44583] Saving new best policy, reward=156.280! [2023-10-12 20:04:13,147][44959] Updated weights for policy 1, policy_version 3140 (0.0008) [2023-10-12 20:04:13,512][44959] Updated weights for policy 1, policy_version 3150 (0.0009) [2023-10-12 20:04:13,882][44959] Updated weights for policy 1, policy_version 3160 (0.0011) [2023-10-12 20:04:15,547][44958] Updated weights for policy 0, policy_version 3140 (0.0010) [2023-10-12 20:04:15,917][44958] Updated weights for policy 0, policy_version 3150 (0.0008) [2023-10-12 20:04:16,289][44958] Updated weights for policy 0, policy_version 3160 (0.0008) [2023-10-12 20:04:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 12561.0, 300 sec: 12996.1). Total num frames: 6455296. Throughput: 0: 1637.2, 1: 1659.5. Samples: 1629492. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-12 20:04:16,444][43579] Avg episode reward: [(0, '132.940'), (1, '158.060')] [2023-10-12 20:04:16,454][44583] Saving new best policy, reward=158.060! [2023-10-12 20:04:18,009][44959] Updated weights for policy 1, policy_version 3170 (0.0009) [2023-10-12 20:04:18,382][44959] Updated weights for policy 1, policy_version 3180 (0.0009) [2023-10-12 20:04:18,747][44959] Updated weights for policy 1, policy_version 3190 (0.0008) [2023-10-12 20:04:19,115][44959] Updated weights for policy 1, policy_version 3200 (0.0007) [2023-10-12 20:04:20,450][44958] Updated weights for policy 0, policy_version 3170 (0.0007) [2023-10-12 20:04:20,833][44958] Updated weights for policy 0, policy_version 3180 (0.0007) [2023-10-12 20:04:21,207][44958] Updated weights for policy 0, policy_version 3190 (0.0009) [2023-10-12 20:04:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 6520832. Throughput: 0: 1635.3, 1: 1646.0. Samples: 1639246. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-12 20:04:21,444][43579] Avg episode reward: [(0, '130.920'), (1, '159.650')] [2023-10-12 20:04:21,445][44583] Saving new best policy, reward=159.650! [2023-10-12 20:04:21,589][44958] Updated weights for policy 0, policy_version 3200 (0.0007) [2023-10-12 20:04:23,334][44959] Updated weights for policy 1, policy_version 3210 (0.0008) [2023-10-12 20:04:23,705][44959] Updated weights for policy 1, policy_version 3220 (0.0009) [2023-10-12 20:04:24,070][44959] Updated weights for policy 1, policy_version 3230 (0.0008) [2023-10-12 20:04:25,746][44958] Updated weights for policy 0, policy_version 3210 (0.0007) [2023-10-12 20:04:26,118][44958] Updated weights for policy 0, policy_version 3220 (0.0008) [2023-10-12 20:04:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 6586368. Throughput: 0: 1639.7, 1: 1661.2. Samples: 1659528. Policy #0 lag: (min: 29.0, avg: 30.2, max: 53.0) [2023-10-12 20:04:26,444][43579] Avg episode reward: [(0, '131.230'), (1, '159.450')] [2023-10-12 20:04:26,491][44958] Updated weights for policy 0, policy_version 3230 (0.0008) [2023-10-12 20:04:28,170][44959] Updated weights for policy 1, policy_version 3240 (0.0008) [2023-10-12 20:04:28,546][44959] Updated weights for policy 1, policy_version 3250 (0.0008) [2023-10-12 20:04:28,918][44959] Updated weights for policy 1, policy_version 3260 (0.0008) [2023-10-12 20:04:30,614][44958] Updated weights for policy 0, policy_version 3240 (0.0010) [2023-10-12 20:04:30,990][44958] Updated weights for policy 0, policy_version 3250 (0.0010) [2023-10-12 20:04:31,372][44958] Updated weights for policy 0, policy_version 3260 (0.0010) [2023-10-12 20:04:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 6651904. Throughput: 0: 1637.1, 1: 1665.9. Samples: 1678926. Policy #0 lag: (min: 29.0, avg: 30.2, max: 53.0) [2023-10-12 20:04:31,443][43579] Avg episode reward: [(0, '134.760'), (1, '160.840')] [2023-10-12 20:04:31,455][44583] Saving new best policy, reward=160.840! [2023-10-12 20:04:31,514][44518] Saving new best policy, reward=134.760! [2023-10-12 20:04:32,951][44959] Updated weights for policy 1, policy_version 3270 (0.0007) [2023-10-12 20:04:33,314][44959] Updated weights for policy 1, policy_version 3280 (0.0008) [2023-10-12 20:04:33,689][44959] Updated weights for policy 1, policy_version 3290 (0.0011) [2023-10-12 20:04:35,579][44958] Updated weights for policy 0, policy_version 3270 (0.0009) [2023-10-12 20:04:35,950][44958] Updated weights for policy 0, policy_version 3280 (0.0011) [2023-10-12 20:04:36,326][44958] Updated weights for policy 0, policy_version 3290 (0.0009) [2023-10-12 20:04:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6717440. Throughput: 0: 1642.5, 1: 1655.1. Samples: 1688714. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) [2023-10-12 20:04:36,443][43579] Avg episode reward: [(0, '138.030'), (1, '163.750')] [2023-10-12 20:04:36,444][44583] Saving new best policy, reward=163.750! [2023-10-12 20:04:36,543][44518] Saving new best policy, reward=138.030! [2023-10-12 20:04:37,901][44959] Updated weights for policy 1, policy_version 3300 (0.0010) [2023-10-12 20:04:38,271][44959] Updated weights for policy 1, policy_version 3310 (0.0007) [2023-10-12 20:04:38,640][44959] Updated weights for policy 1, policy_version 3320 (0.0009) [2023-10-12 20:04:40,345][44958] Updated weights for policy 0, policy_version 3300 (0.0009) [2023-10-12 20:04:40,722][44958] Updated weights for policy 0, policy_version 3310 (0.0010) [2023-10-12 20:04:41,104][44958] Updated weights for policy 0, policy_version 3320 (0.0010) [2023-10-12 20:04:41,443][43579] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 6815744. Throughput: 0: 1638.5, 1: 1666.6. Samples: 1709058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:04:41,444][43579] Avg episode reward: [(0, '138.840'), (1, '164.420')] [2023-10-12 20:04:41,445][44518] Saving new best policy, reward=138.840! [2023-10-12 20:04:41,445][44583] Saving new best policy, reward=164.420! [2023-10-12 20:04:42,707][44959] Updated weights for policy 1, policy_version 3330 (0.0010) [2023-10-12 20:04:43,134][44959] Updated weights for policy 1, policy_version 3340 (0.0010) [2023-10-12 20:04:43,511][44959] Updated weights for policy 1, policy_version 3350 (0.0009) [2023-10-12 20:04:43,873][44959] Updated weights for policy 1, policy_version 3360 (0.0010) [2023-10-12 20:04:45,385][44958] Updated weights for policy 0, policy_version 3330 (0.0008) [2023-10-12 20:04:45,756][44958] Updated weights for policy 0, policy_version 3340 (0.0010) [2023-10-12 20:04:46,125][44958] Updated weights for policy 0, policy_version 3350 (0.0008) [2023-10-12 20:04:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 6848512. Throughput: 0: 1639.5, 1: 1664.9. Samples: 1728502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:04:46,443][43579] Avg episode reward: [(0, '137.260'), (1, '163.390')] [2023-10-12 20:04:46,514][44958] Updated weights for policy 0, policy_version 3360 (0.0010) [2023-10-12 20:04:47,879][44959] Updated weights for policy 1, policy_version 3370 (0.0007) [2023-10-12 20:04:48,245][44959] Updated weights for policy 1, policy_version 3380 (0.0009) [2023-10-12 20:04:48,626][44959] Updated weights for policy 1, policy_version 3390 (0.0010) [2023-10-12 20:04:50,783][44958] Updated weights for policy 0, policy_version 3370 (0.0007) [2023-10-12 20:04:51,157][44958] Updated weights for policy 0, policy_version 3380 (0.0007) [2023-10-12 20:04:51,442][43579] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 6914048. Throughput: 0: 1636.9, 1: 1659.2. Samples: 1738098. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-12 20:04:51,443][43579] Avg episode reward: [(0, '140.910'), (1, '166.380')] [2023-10-12 20:04:51,444][44583] Saving new best policy, reward=166.380! [2023-10-12 20:04:51,530][44958] Updated weights for policy 0, policy_version 3390 (0.0007) [2023-10-12 20:04:51,602][44518] Saving new best policy, reward=140.910! [2023-10-12 20:04:52,667][44959] Updated weights for policy 1, policy_version 3400 (0.0009) [2023-10-12 20:04:53,032][44959] Updated weights for policy 1, policy_version 3410 (0.0009) [2023-10-12 20:04:53,407][44959] Updated weights for policy 1, policy_version 3420 (0.0007) [2023-10-12 20:04:55,711][44958] Updated weights for policy 0, policy_version 3400 (0.0009) [2023-10-12 20:04:56,070][44958] Updated weights for policy 0, policy_version 3410 (0.0009) [2023-10-12 20:04:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 6979584. Throughput: 0: 1638.3, 1: 1657.8. Samples: 1758284. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-12 20:04:56,444][43579] Avg episode reward: [(0, '142.630'), (1, '163.240')] [2023-10-12 20:04:56,448][44958] Updated weights for policy 0, policy_version 3420 (0.0009) [2023-10-12 20:04:56,593][44518] Saving new best policy, reward=142.630! [2023-10-12 20:04:57,499][44959] Updated weights for policy 1, policy_version 3430 (0.0008) [2023-10-12 20:04:57,872][44959] Updated weights for policy 1, policy_version 3440 (0.0007) [2023-10-12 20:04:58,241][44959] Updated weights for policy 1, policy_version 3450 (0.0007) [2023-10-12 20:05:00,562][44958] Updated weights for policy 0, policy_version 3430 (0.0009) [2023-10-12 20:05:00,942][44958] Updated weights for policy 0, policy_version 3440 (0.0009) [2023-10-12 20:05:01,307][44958] Updated weights for policy 0, policy_version 3450 (0.0008) [2023-10-12 20:05:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7045120. Throughput: 0: 1638.9, 1: 1655.2. Samples: 1777724. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-12 20:05:01,443][43579] Avg episode reward: [(0, '138.900'), (1, '165.110')] [2023-10-12 20:05:02,545][44959] Updated weights for policy 1, policy_version 3460 (0.0008) [2023-10-12 20:05:02,915][44959] Updated weights for policy 1, policy_version 3470 (0.0009) [2023-10-12 20:05:03,283][44959] Updated weights for policy 1, policy_version 3480 (0.0009) [2023-10-12 20:05:05,438][44958] Updated weights for policy 0, policy_version 3460 (0.0008) [2023-10-12 20:05:05,810][44958] Updated weights for policy 0, policy_version 3470 (0.0009) [2023-10-12 20:05:06,194][44958] Updated weights for policy 0, policy_version 3480 (0.0010) [2023-10-12 20:05:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7110656. Throughput: 0: 1639.3, 1: 1656.0. Samples: 1787534. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-12 20:05:06,443][43579] Avg episode reward: [(0, '141.310'), (1, '162.610')] [2023-10-12 20:05:07,524][44959] Updated weights for policy 1, policy_version 3490 (0.0008) [2023-10-12 20:05:07,901][44959] Updated weights for policy 1, policy_version 3500 (0.0007) [2023-10-12 20:05:08,261][44959] Updated weights for policy 1, policy_version 3510 (0.0008) [2023-10-12 20:05:08,627][44959] Updated weights for policy 1, policy_version 3520 (0.0008) [2023-10-12 20:05:10,442][44958] Updated weights for policy 0, policy_version 3490 (0.0009) [2023-10-12 20:05:10,818][44958] Updated weights for policy 0, policy_version 3500 (0.0008) [2023-10-12 20:05:11,192][44958] Updated weights for policy 0, policy_version 3510 (0.0010) [2023-10-12 20:05:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7176192. Throughput: 0: 1638.0, 1: 1661.2. Samples: 1807994. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 20:05:11,444][43579] Avg episode reward: [(0, '141.650'), (1, '164.240')] [2023-10-12 20:05:11,572][44958] Updated weights for policy 0, policy_version 3520 (0.0007) [2023-10-12 20:05:12,878][44959] Updated weights for policy 1, policy_version 3530 (0.0008) [2023-10-12 20:05:13,243][44959] Updated weights for policy 1, policy_version 3540 (0.0007) [2023-10-12 20:05:13,607][44959] Updated weights for policy 1, policy_version 3550 (0.0008) [2023-10-12 20:05:15,617][44958] Updated weights for policy 0, policy_version 3530 (0.0007) [2023-10-12 20:05:15,997][44958] Updated weights for policy 0, policy_version 3540 (0.0008) [2023-10-12 20:05:16,378][44958] Updated weights for policy 0, policy_version 3550 (0.0009) [2023-10-12 20:05:16,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 7241728. Throughput: 0: 1641.6, 1: 1656.9. Samples: 1827360. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-12 20:05:16,444][43579] Avg episode reward: [(0, '147.670'), (1, '164.630')] [2023-10-12 20:05:16,455][44518] Saving new best policy, reward=147.670! [2023-10-12 20:05:17,830][44959] Updated weights for policy 1, policy_version 3560 (0.0007) [2023-10-12 20:05:18,190][44959] Updated weights for policy 1, policy_version 3570 (0.0007) [2023-10-12 20:05:18,563][44959] Updated weights for policy 1, policy_version 3580 (0.0008) [2023-10-12 20:05:20,520][44958] Updated weights for policy 0, policy_version 3560 (0.0008) [2023-10-12 20:05:20,898][44958] Updated weights for policy 0, policy_version 3570 (0.0009) [2023-10-12 20:05:21,266][44958] Updated weights for policy 0, policy_version 3580 (0.0008) [2023-10-12 20:05:21,442][43579] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13107.2). Total num frames: 7340032. Throughput: 0: 1642.8, 1: 1654.6. Samples: 1837098. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-12 20:05:21,443][43579] Avg episode reward: [(0, '147.700'), (1, '168.360')] [2023-10-12 20:05:21,444][44518] Saving new best policy, reward=147.700! [2023-10-12 20:05:21,444][44583] Saving new best policy, reward=168.360! [2023-10-12 20:05:22,668][44959] Updated weights for policy 1, policy_version 3590 (0.0008) [2023-10-12 20:05:23,041][44959] Updated weights for policy 1, policy_version 3600 (0.0010) [2023-10-12 20:05:23,419][44959] Updated weights for policy 1, policy_version 3610 (0.0010) [2023-10-12 20:05:25,633][44958] Updated weights for policy 0, policy_version 3590 (0.0010) [2023-10-12 20:05:26,014][44958] Updated weights for policy 0, policy_version 3600 (0.0008) [2023-10-12 20:05:26,383][44958] Updated weights for policy 0, policy_version 3610 (0.0008) [2023-10-12 20:05:26,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7372800. Throughput: 0: 1637.8, 1: 1651.4. Samples: 1857072. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-12 20:05:26,443][43579] Avg episode reward: [(0, '150.500'), (1, '166.930')] [2023-10-12 20:05:26,595][44518] Saving new best policy, reward=150.500! [2023-10-12 20:05:27,796][44959] Updated weights for policy 1, policy_version 3620 (0.0007) [2023-10-12 20:05:28,192][44959] Updated weights for policy 1, policy_version 3630 (0.0007) [2023-10-12 20:05:28,557][44959] Updated weights for policy 1, policy_version 3640 (0.0008) [2023-10-12 20:05:30,521][44958] Updated weights for policy 0, policy_version 3620 (0.0010) [2023-10-12 20:05:30,891][44958] Updated weights for policy 0, policy_version 3630 (0.0008) [2023-10-12 20:05:31,266][44958] Updated weights for policy 0, policy_version 3640 (0.0009) [2023-10-12 20:05:31,443][43579] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7438336. Throughput: 0: 1642.0, 1: 1648.4. Samples: 1876572. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-12 20:05:31,443][43579] Avg episode reward: [(0, '153.360'), (1, '170.340')] [2023-10-12 20:05:31,453][44583] Saving new best policy, reward=170.340! [2023-10-12 20:05:31,556][44518] Saving new best policy, reward=153.360! [2023-10-12 20:05:32,527][44959] Updated weights for policy 1, policy_version 3650 (0.0007) [2023-10-12 20:05:32,897][44959] Updated weights for policy 1, policy_version 3660 (0.0008) [2023-10-12 20:05:33,263][44959] Updated weights for policy 1, policy_version 3670 (0.0008) [2023-10-12 20:05:33,632][44959] Updated weights for policy 1, policy_version 3680 (0.0008) [2023-10-12 20:05:35,427][44958] Updated weights for policy 0, policy_version 3650 (0.0008) [2023-10-12 20:05:35,798][44958] Updated weights for policy 0, policy_version 3660 (0.0010) [2023-10-12 20:05:36,172][44958] Updated weights for policy 0, policy_version 3670 (0.0007) [2023-10-12 20:05:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7503872. Throughput: 0: 1642.9, 1: 1646.0. Samples: 1886096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:05:36,443][43579] Avg episode reward: [(0, '157.160'), (1, '168.720')] [2023-10-12 20:05:36,542][44518] Saving new best policy, reward=157.160! [2023-10-12 20:05:36,545][44958] Updated weights for policy 0, policy_version 3680 (0.0008) [2023-10-12 20:05:37,755][44959] Updated weights for policy 1, policy_version 3690 (0.0009) [2023-10-12 20:05:38,121][44959] Updated weights for policy 1, policy_version 3700 (0.0009) [2023-10-12 20:05:38,483][44959] Updated weights for policy 1, policy_version 3710 (0.0009) [2023-10-12 20:05:40,738][44958] Updated weights for policy 0, policy_version 3690 (0.0008) [2023-10-12 20:05:41,108][44958] Updated weights for policy 0, policy_version 3700 (0.0008) [2023-10-12 20:05:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 7569408. Throughput: 0: 1636.9, 1: 1648.8. Samples: 1906138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:05:41,443][43579] Avg episode reward: [(0, '155.320'), (1, '167.050')] [2023-10-12 20:05:41,484][44958] Updated weights for policy 0, policy_version 3710 (0.0009) [2023-10-12 20:05:42,672][44959] Updated weights for policy 1, policy_version 3720 (0.0009) [2023-10-12 20:05:43,049][44959] Updated weights for policy 1, policy_version 3730 (0.0009) [2023-10-12 20:05:43,416][44959] Updated weights for policy 1, policy_version 3740 (0.0008) [2023-10-12 20:05:45,584][44958] Updated weights for policy 0, policy_version 3720 (0.0008) [2023-10-12 20:05:45,960][44958] Updated weights for policy 0, policy_version 3730 (0.0008) [2023-10-12 20:05:46,339][44958] Updated weights for policy 0, policy_version 3740 (0.0008) [2023-10-12 20:05:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7634944. Throughput: 0: 1639.5, 1: 1655.4. Samples: 1925994. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-12 20:05:46,444][43579] Avg episode reward: [(0, '154.170'), (1, '165.330')] [2023-10-12 20:05:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000003744_3833856.pth... [2023-10-12 20:05:46,478][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000003744_3833856.pth... [2023-10-12 20:05:46,482][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000002208_2260992.pth [2023-10-12 20:05:46,515][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000002208_2260992.pth [2023-10-12 20:05:47,348][44959] Updated weights for policy 1, policy_version 3750 (0.0009) [2023-10-12 20:05:47,714][44959] Updated weights for policy 1, policy_version 3760 (0.0010) [2023-10-12 20:05:48,086][44959] Updated weights for policy 1, policy_version 3770 (0.0009) [2023-10-12 20:05:50,534][44958] Updated weights for policy 0, policy_version 3750 (0.0010) [2023-10-12 20:05:50,897][44958] Updated weights for policy 0, policy_version 3760 (0.0007) [2023-10-12 20:05:51,273][44958] Updated weights for policy 0, policy_version 3770 (0.0010) [2023-10-12 20:05:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7700480. Throughput: 0: 1639.6, 1: 1652.3. Samples: 1935668. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-12 20:05:51,444][43579] Avg episode reward: [(0, '154.440'), (1, '169.400')] [2023-10-12 20:05:52,287][44959] Updated weights for policy 1, policy_version 3780 (0.0008) [2023-10-12 20:05:52,656][44959] Updated weights for policy 1, policy_version 3790 (0.0008) [2023-10-12 20:05:53,014][44959] Updated weights for policy 1, policy_version 3800 (0.0010) [2023-10-12 20:05:55,829][44958] Updated weights for policy 0, policy_version 3780 (0.0010) [2023-10-12 20:05:56,224][44958] Updated weights for policy 0, policy_version 3790 (0.0009) [2023-10-12 20:05:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7766016. Throughput: 0: 1634.8, 1: 1649.2. Samples: 1955772. Policy #0 lag: (min: 16.0, avg: 42.6, max: 48.0) [2023-10-12 20:05:56,444][43579] Avg episode reward: [(0, '156.300'), (1, '169.330')] [2023-10-12 20:05:56,608][44958] Updated weights for policy 0, policy_version 3800 (0.0008) [2023-10-12 20:05:57,158][44959] Updated weights for policy 1, policy_version 3810 (0.0009) [2023-10-12 20:05:57,527][44959] Updated weights for policy 1, policy_version 3820 (0.0009) [2023-10-12 20:05:57,901][44959] Updated weights for policy 1, policy_version 3830 (0.0007) [2023-10-12 20:05:58,258][44959] Updated weights for policy 1, policy_version 3840 (0.0007) [2023-10-12 20:06:00,619][44958] Updated weights for policy 0, policy_version 3810 (0.0008) [2023-10-12 20:06:01,000][44958] Updated weights for policy 0, policy_version 3820 (0.0008) [2023-10-12 20:06:01,372][44958] Updated weights for policy 0, policy_version 3830 (0.0007) [2023-10-12 20:06:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7831552. Throughput: 0: 1640.6, 1: 1652.5. Samples: 1975552. Policy #0 lag: (min: 31.0, avg: 31.8, max: 47.0) [2023-10-12 20:06:01,443][43579] Avg episode reward: [(0, '156.150'), (1, '171.830')] [2023-10-12 20:06:01,452][44583] Saving new best policy, reward=171.830! [2023-10-12 20:06:01,744][44958] Updated weights for policy 0, policy_version 3840 (0.0007) [2023-10-12 20:06:02,338][44959] Updated weights for policy 1, policy_version 3850 (0.0010) [2023-10-12 20:06:02,700][44959] Updated weights for policy 1, policy_version 3860 (0.0007) [2023-10-12 20:06:03,070][44959] Updated weights for policy 1, policy_version 3870 (0.0009) [2023-10-12 20:06:05,715][44958] Updated weights for policy 0, policy_version 3850 (0.0008) [2023-10-12 20:06:06,094][44958] Updated weights for policy 0, policy_version 3860 (0.0007) [2023-10-12 20:06:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7897088. Throughput: 0: 1636.1, 1: 1653.1. Samples: 1985112. Policy #0 lag: (min: 31.0, avg: 31.8, max: 47.0) [2023-10-12 20:06:06,444][43579] Avg episode reward: [(0, '158.040'), (1, '174.840')] [2023-10-12 20:06:06,445][44583] Saving new best policy, reward=174.840! [2023-10-12 20:06:06,472][44958] Updated weights for policy 0, policy_version 3870 (0.0007) [2023-10-12 20:06:06,540][44518] Saving new best policy, reward=158.040! [2023-10-12 20:06:07,174][44959] Updated weights for policy 1, policy_version 3880 (0.0007) [2023-10-12 20:06:07,538][44959] Updated weights for policy 1, policy_version 3890 (0.0010) [2023-10-12 20:06:07,910][44959] Updated weights for policy 1, policy_version 3900 (0.0009) [2023-10-12 20:06:10,692][44958] Updated weights for policy 0, policy_version 3880 (0.0010) [2023-10-12 20:06:11,063][44958] Updated weights for policy 0, policy_version 3890 (0.0009) [2023-10-12 20:06:11,433][44958] Updated weights for policy 0, policy_version 3900 (0.0010) [2023-10-12 20:06:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 7962624. Throughput: 0: 1641.0, 1: 1656.8. Samples: 2005474. Policy #0 lag: (min: 17.0, avg: 35.3, max: 49.0) [2023-10-12 20:06:11,443][43579] Avg episode reward: [(0, '162.850'), (1, '176.900')] [2023-10-12 20:06:11,444][44583] Saving new best policy, reward=176.900! [2023-10-12 20:06:11,584][44518] Saving new best policy, reward=162.850! [2023-10-12 20:06:12,061][44959] Updated weights for policy 1, policy_version 3910 (0.0008) [2023-10-12 20:06:12,439][44959] Updated weights for policy 1, policy_version 3920 (0.0007) [2023-10-12 20:06:12,818][44959] Updated weights for policy 1, policy_version 3930 (0.0007) [2023-10-12 20:06:15,700][44958] Updated weights for policy 0, policy_version 3910 (0.0009) [2023-10-12 20:06:16,079][44958] Updated weights for policy 0, policy_version 3920 (0.0009) [2023-10-12 20:06:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 8028160. Throughput: 0: 1639.6, 1: 1658.1. Samples: 2024968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:06:16,443][43579] Avg episode reward: [(0, '165.520'), (1, '179.680')] [2023-10-12 20:06:16,451][44583] Saving new best policy, reward=179.680! [2023-10-12 20:06:16,462][44958] Updated weights for policy 0, policy_version 3930 (0.0008) [2023-10-12 20:06:16,676][44518] Saving new best policy, reward=165.520! [2023-10-12 20:06:17,004][44959] Updated weights for policy 1, policy_version 3940 (0.0008) [2023-10-12 20:06:17,377][44959] Updated weights for policy 1, policy_version 3950 (0.0007) [2023-10-12 20:06:17,741][44959] Updated weights for policy 1, policy_version 3960 (0.0008) [2023-10-12 20:06:20,708][44958] Updated weights for policy 0, policy_version 3940 (0.0009) [2023-10-12 20:06:21,068][44958] Updated weights for policy 0, policy_version 3950 (0.0009) [2023-10-12 20:06:21,439][44958] Updated weights for policy 0, policy_version 3960 (0.0007) [2023-10-12 20:06:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 8093696. Throughput: 0: 1635.5, 1: 1658.8. Samples: 2034338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:06:21,443][43579] Avg episode reward: [(0, '168.490'), (1, '179.100')] [2023-10-12 20:06:21,738][44518] Saving new best policy, reward=168.490! [2023-10-12 20:06:21,819][44959] Updated weights for policy 1, policy_version 3970 (0.0008) [2023-10-12 20:06:22,196][44959] Updated weights for policy 1, policy_version 3980 (0.0010) [2023-10-12 20:06:22,568][44959] Updated weights for policy 1, policy_version 3990 (0.0008) [2023-10-12 20:06:22,929][44959] Updated weights for policy 1, policy_version 4000 (0.0007) [2023-10-12 20:06:25,367][44958] Updated weights for policy 0, policy_version 3970 (0.0009) [2023-10-12 20:06:25,737][44958] Updated weights for policy 0, policy_version 3980 (0.0011) [2023-10-12 20:06:26,103][44958] Updated weights for policy 0, policy_version 3990 (0.0009) [2023-10-12 20:06:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 8159232. Throughput: 0: 1636.4, 1: 1662.3. Samples: 2054580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:06:26,444][43579] Avg episode reward: [(0, '167.660'), (1, '179.350')] [2023-10-12 20:06:26,485][44958] Updated weights for policy 0, policy_version 4000 (0.0009) [2023-10-12 20:06:27,137][44959] Updated weights for policy 1, policy_version 4010 (0.0007) [2023-10-12 20:06:27,501][44959] Updated weights for policy 1, policy_version 4020 (0.0008) [2023-10-12 20:06:27,871][44959] Updated weights for policy 1, policy_version 4030 (0.0007) [2023-10-12 20:06:30,669][44958] Updated weights for policy 0, policy_version 4010 (0.0010) [2023-10-12 20:06:31,041][44958] Updated weights for policy 0, policy_version 4020 (0.0011) [2023-10-12 20:06:31,406][44958] Updated weights for policy 0, policy_version 4030 (0.0010) [2023-10-12 20:06:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 8224768. Throughput: 0: 1632.9, 1: 1654.7. Samples: 2073934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:06:31,443][43579] Avg episode reward: [(0, '167.340'), (1, '176.880')] [2023-10-12 20:06:32,078][44959] Updated weights for policy 1, policy_version 4040 (0.0007) [2023-10-12 20:06:32,452][44959] Updated weights for policy 1, policy_version 4050 (0.0008) [2023-10-12 20:06:32,817][44959] Updated weights for policy 1, policy_version 4060 (0.0009) [2023-10-12 20:06:35,403][44958] Updated weights for policy 0, policy_version 4040 (0.0008) [2023-10-12 20:06:35,786][44958] Updated weights for policy 0, policy_version 4050 (0.0009) [2023-10-12 20:06:36,147][44958] Updated weights for policy 0, policy_version 4060 (0.0008) [2023-10-12 20:06:36,442][43579] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 8323072. Throughput: 0: 1634.1, 1: 1653.7. Samples: 2083620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:06:36,443][43579] Avg episode reward: [(0, '165.980'), (1, '176.300')] [2023-10-12 20:06:36,977][44959] Updated weights for policy 1, policy_version 4070 (0.0009) [2023-10-12 20:06:37,342][44959] Updated weights for policy 1, policy_version 4080 (0.0007) [2023-10-12 20:06:37,712][44959] Updated weights for policy 1, policy_version 4090 (0.0009) [2023-10-12 20:06:40,453][44958] Updated weights for policy 0, policy_version 4070 (0.0009) [2023-10-12 20:06:40,827][44958] Updated weights for policy 0, policy_version 4080 (0.0009) [2023-10-12 20:06:41,207][44958] Updated weights for policy 0, policy_version 4090 (0.0008) [2023-10-12 20:06:41,442][43579] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 8388608. Throughput: 0: 1635.3, 1: 1656.9. Samples: 2103920. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 20:06:41,443][43579] Avg episode reward: [(0, '165.930'), (1, '176.370')] [2023-10-12 20:06:41,800][44959] Updated weights for policy 1, policy_version 4100 (0.0010) [2023-10-12 20:06:42,168][44959] Updated weights for policy 1, policy_version 4110 (0.0007) [2023-10-12 20:06:42,546][44959] Updated weights for policy 1, policy_version 4120 (0.0008) [2023-10-12 20:06:45,413][44958] Updated weights for policy 0, policy_version 4100 (0.0008) [2023-10-12 20:06:45,785][44958] Updated weights for policy 0, policy_version 4110 (0.0008) [2023-10-12 20:06:46,168][44958] Updated weights for policy 0, policy_version 4120 (0.0008) [2023-10-12 20:06:46,442][43579] Fps is (10 sec: 9830.3, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 8421376. Throughput: 0: 1628.9, 1: 1653.8. Samples: 2123276. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 20:06:46,443][43579] Avg episode reward: [(0, '166.770'), (1, '175.790')] [2023-10-12 20:06:46,679][44959] Updated weights for policy 1, policy_version 4130 (0.0008) [2023-10-12 20:06:47,047][44959] Updated weights for policy 1, policy_version 4140 (0.0009) [2023-10-12 20:06:47,424][44959] Updated weights for policy 1, policy_version 4150 (0.0009) [2023-10-12 20:06:47,794][44959] Updated weights for policy 1, policy_version 4160 (0.0008) [2023-10-12 20:06:50,437][44958] Updated weights for policy 0, policy_version 4130 (0.0008) [2023-10-12 20:06:50,808][44958] Updated weights for policy 0, policy_version 4140 (0.0008) [2023-10-12 20:06:51,176][44958] Updated weights for policy 0, policy_version 4150 (0.0008) [2023-10-12 20:06:51,442][43579] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 8486912. Throughput: 0: 1627.0, 1: 1654.5. Samples: 2132776. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 20:06:51,443][43579] Avg episode reward: [(0, '165.170'), (1, '179.050')] [2023-10-12 20:06:51,548][44958] Updated weights for policy 0, policy_version 4160 (0.0007) [2023-10-12 20:06:52,087][44959] Updated weights for policy 1, policy_version 4170 (0.0008) [2023-10-12 20:06:52,461][44959] Updated weights for policy 1, policy_version 4180 (0.0008) [2023-10-12 20:06:52,828][44959] Updated weights for policy 1, policy_version 4190 (0.0007) [2023-10-12 20:06:55,796][44958] Updated weights for policy 0, policy_version 4170 (0.0008) [2023-10-12 20:06:56,171][44958] Updated weights for policy 0, policy_version 4180 (0.0008) [2023-10-12 20:06:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 8552448. Throughput: 0: 1631.1, 1: 1652.0. Samples: 2153214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:06:56,443][43579] Avg episode reward: [(0, '170.260'), (1, '181.260')] [2023-10-12 20:06:56,444][44583] Saving new best policy, reward=181.260! [2023-10-12 20:06:56,543][44958] Updated weights for policy 0, policy_version 4190 (0.0010) [2023-10-12 20:06:56,616][44518] Saving new best policy, reward=170.260! [2023-10-12 20:06:56,908][44959] Updated weights for policy 1, policy_version 4200 (0.0009) [2023-10-12 20:06:57,274][44959] Updated weights for policy 1, policy_version 4210 (0.0008) [2023-10-12 20:06:57,642][44959] Updated weights for policy 1, policy_version 4220 (0.0011) [2023-10-12 20:07:00,608][44958] Updated weights for policy 0, policy_version 4200 (0.0010) [2023-10-12 20:07:00,989][44958] Updated weights for policy 0, policy_version 4210 (0.0009) [2023-10-12 20:07:01,362][44958] Updated weights for policy 0, policy_version 4220 (0.0008) [2023-10-12 20:07:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 8617984. Throughput: 0: 1627.2, 1: 1651.9. Samples: 2172528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:07:01,443][43579] Avg episode reward: [(0, '174.120'), (1, '179.390')] [2023-10-12 20:07:01,506][44518] Saving new best policy, reward=174.120! [2023-10-12 20:07:01,685][44959] Updated weights for policy 1, policy_version 4230 (0.0009) [2023-10-12 20:07:02,051][44959] Updated weights for policy 1, policy_version 4240 (0.0007) [2023-10-12 20:07:02,425][44959] Updated weights for policy 1, policy_version 4250 (0.0008) [2023-10-12 20:07:05,654][44958] Updated weights for policy 0, policy_version 4230 (0.0008) [2023-10-12 20:07:06,025][44958] Updated weights for policy 0, policy_version 4240 (0.0010) [2023-10-12 20:07:06,398][44958] Updated weights for policy 0, policy_version 4250 (0.0008) [2023-10-12 20:07:06,414][44959] Updated weights for policy 1, policy_version 4260 (0.0009) [2023-10-12 20:07:06,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 8683520. Throughput: 0: 1634.2, 1: 1653.7. Samples: 2182294. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 20:07:06,444][43579] Avg episode reward: [(0, '172.440'), (1, '176.000')] [2023-10-12 20:07:06,785][44959] Updated weights for policy 1, policy_version 4270 (0.0008) [2023-10-12 20:07:07,150][44959] Updated weights for policy 1, policy_version 4280 (0.0009) [2023-10-12 20:07:10,433][44958] Updated weights for policy 0, policy_version 4260 (0.0008) [2023-10-12 20:07:10,800][44958] Updated weights for policy 0, policy_version 4270 (0.0010) [2023-10-12 20:07:11,184][44958] Updated weights for policy 0, policy_version 4280 (0.0008) [2023-10-12 20:07:11,373][44959] Updated weights for policy 1, policy_version 4290 (0.0008) [2023-10-12 20:07:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 8749056. Throughput: 0: 1639.0, 1: 1658.1. Samples: 2202952. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 20:07:11,443][43579] Avg episode reward: [(0, '172.090'), (1, '184.740')] [2023-10-12 20:07:11,735][44959] Updated weights for policy 1, policy_version 4300 (0.0008) [2023-10-12 20:07:12,108][44959] Updated weights for policy 1, policy_version 4310 (0.0010) [2023-10-12 20:07:12,473][44959] Updated weights for policy 1, policy_version 4320 (0.0010) [2023-10-12 20:07:12,474][44583] Saving new best policy, reward=184.740! [2023-10-12 20:07:15,447][44958] Updated weights for policy 0, policy_version 4290 (0.0009) [2023-10-12 20:07:15,822][44958] Updated weights for policy 0, policy_version 4300 (0.0009) [2023-10-12 20:07:16,194][44958] Updated weights for policy 0, policy_version 4310 (0.0008) [2023-10-12 20:07:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 8814592. Throughput: 0: 1641.8, 1: 1660.2. Samples: 2222526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:07:16,444][43579] Avg episode reward: [(0, '170.020'), (1, '177.770')] [2023-10-12 20:07:16,564][44958] Updated weights for policy 0, policy_version 4320 (0.0008) [2023-10-12 20:07:16,651][44959] Updated weights for policy 1, policy_version 4330 (0.0008) [2023-10-12 20:07:17,020][44959] Updated weights for policy 1, policy_version 4340 (0.0009) [2023-10-12 20:07:17,385][44959] Updated weights for policy 1, policy_version 4350 (0.0010) [2023-10-12 20:07:20,773][44958] Updated weights for policy 0, policy_version 4330 (0.0010) [2023-10-12 20:07:21,148][44958] Updated weights for policy 0, policy_version 4340 (0.0009) [2023-10-12 20:07:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.3). Total num frames: 8880128. Throughput: 0: 1634.2, 1: 1660.5. Samples: 2231880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:07:21,443][43579] Avg episode reward: [(0, '172.290'), (1, '173.350')] [2023-10-12 20:07:21,525][44959] Updated weights for policy 1, policy_version 4360 (0.0007) [2023-10-12 20:07:21,526][44958] Updated weights for policy 0, policy_version 4350 (0.0008) [2023-10-12 20:07:21,890][44959] Updated weights for policy 1, policy_version 4370 (0.0009) [2023-10-12 20:07:22,271][44959] Updated weights for policy 1, policy_version 4380 (0.0009) [2023-10-12 20:07:25,654][44958] Updated weights for policy 0, policy_version 4360 (0.0009) [2023-10-12 20:07:26,029][44958] Updated weights for policy 0, policy_version 4370 (0.0009) [2023-10-12 20:07:26,405][44958] Updated weights for policy 0, policy_version 4380 (0.0007) [2023-10-12 20:07:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 8945664. Throughput: 0: 1640.0, 1: 1654.0. Samples: 2252152. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 20:07:26,443][43579] Avg episode reward: [(0, '171.640'), (1, '177.250')] [2023-10-12 20:07:26,488][44959] Updated weights for policy 1, policy_version 4390 (0.0009) [2023-10-12 20:07:26,858][44959] Updated weights for policy 1, policy_version 4400 (0.0008) [2023-10-12 20:07:27,222][44959] Updated weights for policy 1, policy_version 4410 (0.0008) [2023-10-12 20:07:30,714][44958] Updated weights for policy 0, policy_version 4390 (0.0009) [2023-10-12 20:07:31,100][44958] Updated weights for policy 0, policy_version 4400 (0.0010) [2023-10-12 20:07:31,320][44959] Updated weights for policy 1, policy_version 4420 (0.0007) [2023-10-12 20:07:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9011200. Throughput: 0: 1639.1, 1: 1656.2. Samples: 2271564. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 20:07:31,443][43579] Avg episode reward: [(0, '171.310'), (1, '180.550')] [2023-10-12 20:07:31,465][44958] Updated weights for policy 0, policy_version 4410 (0.0010) [2023-10-12 20:07:31,682][44959] Updated weights for policy 1, policy_version 4430 (0.0007) [2023-10-12 20:07:32,045][44959] Updated weights for policy 1, policy_version 4440 (0.0009) [2023-10-12 20:07:35,457][44958] Updated weights for policy 0, policy_version 4420 (0.0009) [2023-10-12 20:07:35,829][44958] Updated weights for policy 0, policy_version 4430 (0.0008) [2023-10-12 20:07:36,171][44959] Updated weights for policy 1, policy_version 4450 (0.0010) [2023-10-12 20:07:36,199][44958] Updated weights for policy 0, policy_version 4440 (0.0009) [2023-10-12 20:07:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 12561.0, 300 sec: 12996.2). Total num frames: 9076736. Throughput: 0: 1638.1, 1: 1655.8. Samples: 2281000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:07:36,444][43579] Avg episode reward: [(0, '172.540'), (1, '185.270')] [2023-10-12 20:07:36,545][44959] Updated weights for policy 1, policy_version 4460 (0.0009) [2023-10-12 20:07:36,914][44959] Updated weights for policy 1, policy_version 4470 (0.0008) [2023-10-12 20:07:37,289][44583] Saving new best policy, reward=185.270! [2023-10-12 20:07:37,293][44959] Updated weights for policy 1, policy_version 4480 (0.0011) [2023-10-12 20:07:40,596][44958] Updated weights for policy 0, policy_version 4450 (0.0009) [2023-10-12 20:07:40,962][44958] Updated weights for policy 0, policy_version 4460 (0.0011) [2023-10-12 20:07:41,345][44958] Updated weights for policy 0, policy_version 4470 (0.0008) [2023-10-12 20:07:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 9142272. Throughput: 0: 1631.9, 1: 1659.8. Samples: 2301340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:07:41,443][43579] Avg episode reward: [(0, '171.380'), (1, '182.710')] [2023-10-12 20:07:41,470][44959] Updated weights for policy 1, policy_version 4490 (0.0009) [2023-10-12 20:07:41,717][44958] Updated weights for policy 0, policy_version 4480 (0.0008) [2023-10-12 20:07:41,833][44959] Updated weights for policy 1, policy_version 4500 (0.0008) [2023-10-12 20:07:42,207][44959] Updated weights for policy 1, policy_version 4510 (0.0008) [2023-10-12 20:07:45,901][44958] Updated weights for policy 0, policy_version 4490 (0.0008) [2023-10-12 20:07:46,266][44958] Updated weights for policy 0, policy_version 4500 (0.0010) [2023-10-12 20:07:46,359][44959] Updated weights for policy 1, policy_version 4520 (0.0007) [2023-10-12 20:07:46,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9207808. Throughput: 0: 1635.2, 1: 1657.8. Samples: 2320716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:07:46,443][43579] Avg episode reward: [(0, '172.650'), (1, '189.710')] [2023-10-12 20:07:46,634][44958] Updated weights for policy 0, policy_version 4510 (0.0007) [2023-10-12 20:07:46,708][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000004512_4620288.pth... [2023-10-12 20:07:46,746][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000002976_3047424.pth [2023-10-12 20:07:46,751][44959] Updated weights for policy 1, policy_version 4530 (0.0009) [2023-10-12 20:07:47,117][44959] Updated weights for policy 1, policy_version 4540 (0.0008) [2023-10-12 20:07:47,265][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000004544_4653056.pth... [2023-10-12 20:07:47,304][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000002976_3047424.pth [2023-10-12 20:07:47,309][44583] Saving new best policy, reward=189.710! [2023-10-12 20:07:50,968][44958] Updated weights for policy 0, policy_version 4520 (0.0007) [2023-10-12 20:07:51,342][44959] Updated weights for policy 1, policy_version 4550 (0.0008) [2023-10-12 20:07:51,345][44958] Updated weights for policy 0, policy_version 4530 (0.0008) [2023-10-12 20:07:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9273344. Throughput: 0: 1627.3, 1: 1655.1. Samples: 2330000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:07:51,443][43579] Avg episode reward: [(0, '173.600'), (1, '191.280')] [2023-10-12 20:07:51,708][44958] Updated weights for policy 0, policy_version 4540 (0.0008) [2023-10-12 20:07:51,714][44959] Updated weights for policy 1, policy_version 4560 (0.0009) [2023-10-12 20:07:52,074][44959] Updated weights for policy 1, policy_version 4570 (0.0010) [2023-10-12 20:07:52,297][44583] Saving new best policy, reward=191.280! [2023-10-12 20:07:55,892][44958] Updated weights for policy 0, policy_version 4550 (0.0009) [2023-10-12 20:07:56,260][44958] Updated weights for policy 0, policy_version 4560 (0.0009) [2023-10-12 20:07:56,274][44959] Updated weights for policy 1, policy_version 4580 (0.0009) [2023-10-12 20:07:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 9338880. Throughput: 0: 1629.1, 1: 1648.7. Samples: 2350452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:07:56,444][43579] Avg episode reward: [(0, '171.760'), (1, '191.750')] [2023-10-12 20:07:56,637][44958] Updated weights for policy 0, policy_version 4570 (0.0009) [2023-10-12 20:07:56,643][44959] Updated weights for policy 1, policy_version 4590 (0.0008) [2023-10-12 20:07:57,013][44959] Updated weights for policy 1, policy_version 4600 (0.0009) [2023-10-12 20:07:57,305][44583] Saving new best policy, reward=191.750! [2023-10-12 20:08:00,812][44958] Updated weights for policy 0, policy_version 4580 (0.0009) [2023-10-12 20:08:01,190][44959] Updated weights for policy 1, policy_version 4610 (0.0009) [2023-10-12 20:08:01,193][44958] Updated weights for policy 0, policy_version 4590 (0.0008) [2023-10-12 20:08:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9404416. Throughput: 0: 1633.3, 1: 1644.4. Samples: 2370020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:08:01,443][43579] Avg episode reward: [(0, '171.940'), (1, '190.720')] [2023-10-12 20:08:01,553][44958] Updated weights for policy 0, policy_version 4600 (0.0009) [2023-10-12 20:08:01,562][44959] Updated weights for policy 1, policy_version 4620 (0.0008) [2023-10-12 20:08:01,937][44959] Updated weights for policy 1, policy_version 4630 (0.0009) [2023-10-12 20:08:02,297][44959] Updated weights for policy 1, policy_version 4640 (0.0010) [2023-10-12 20:08:05,694][44958] Updated weights for policy 0, policy_version 4610 (0.0008) [2023-10-12 20:08:06,067][44958] Updated weights for policy 0, policy_version 4620 (0.0009) [2023-10-12 20:08:06,387][44959] Updated weights for policy 1, policy_version 4650 (0.0009) [2023-10-12 20:08:06,428][44958] Updated weights for policy 0, policy_version 4630 (0.0007) [2023-10-12 20:08:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 9469952. Throughput: 0: 1632.6, 1: 1646.3. Samples: 2379428. Policy #0 lag: (min: 10.0, avg: 13.1, max: 42.0) [2023-10-12 20:08:06,443][43579] Avg episode reward: [(0, '174.320'), (1, '191.150')] [2023-10-12 20:08:06,750][44959] Updated weights for policy 1, policy_version 4660 (0.0008) [2023-10-12 20:08:06,803][44518] Saving new best policy, reward=174.320! [2023-10-12 20:08:06,807][44958] Updated weights for policy 0, policy_version 4640 (0.0010) [2023-10-12 20:08:07,123][44959] Updated weights for policy 1, policy_version 4670 (0.0008) [2023-10-12 20:08:10,975][44958] Updated weights for policy 0, policy_version 4650 (0.0010) [2023-10-12 20:08:11,346][44958] Updated weights for policy 0, policy_version 4660 (0.0008) [2023-10-12 20:08:11,420][44959] Updated weights for policy 1, policy_version 4680 (0.0010) [2023-10-12 20:08:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9535488. Throughput: 0: 1629.9, 1: 1649.5. Samples: 2399726. Policy #0 lag: (min: 10.0, avg: 13.1, max: 42.0) [2023-10-12 20:08:11,443][43579] Avg episode reward: [(0, '174.850'), (1, '191.400')] [2023-10-12 20:08:11,721][44958] Updated weights for policy 0, policy_version 4670 (0.0007) [2023-10-12 20:08:11,781][44959] Updated weights for policy 1, policy_version 4690 (0.0009) [2023-10-12 20:08:11,792][44518] Saving new best policy, reward=174.850! [2023-10-12 20:08:12,152][44959] Updated weights for policy 1, policy_version 4700 (0.0008) [2023-10-12 20:08:16,207][44958] Updated weights for policy 0, policy_version 4680 (0.0010) [2023-10-12 20:08:16,307][44959] Updated weights for policy 1, policy_version 4710 (0.0009) [2023-10-12 20:08:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9601024. Throughput: 0: 1635.9, 1: 1642.7. Samples: 2419106. Policy #0 lag: (min: 31.0, avg: 32.3, max: 57.0) [2023-10-12 20:08:16,444][43579] Avg episode reward: [(0, '174.640'), (1, '190.270')] [2023-10-12 20:08:16,592][44958] Updated weights for policy 0, policy_version 4690 (0.0007) [2023-10-12 20:08:16,675][44959] Updated weights for policy 1, policy_version 4720 (0.0010) [2023-10-12 20:08:16,963][44958] Updated weights for policy 0, policy_version 4700 (0.0008) [2023-10-12 20:08:17,033][44959] Updated weights for policy 1, policy_version 4730 (0.0008) [2023-10-12 20:08:20,803][44958] Updated weights for policy 0, policy_version 4710 (0.0007) [2023-10-12 20:08:20,988][44959] Updated weights for policy 1, policy_version 4740 (0.0010) [2023-10-12 20:08:21,186][44958] Updated weights for policy 0, policy_version 4720 (0.0008) [2023-10-12 20:08:21,367][44959] Updated weights for policy 1, policy_version 4750 (0.0008) [2023-10-12 20:08:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9666560. Throughput: 0: 1629.9, 1: 1648.0. Samples: 2428508. Policy #0 lag: (min: 31.0, avg: 32.3, max: 57.0) [2023-10-12 20:08:21,444][43579] Avg episode reward: [(0, '176.720'), (1, '189.820')] [2023-10-12 20:08:21,551][44958] Updated weights for policy 0, policy_version 4730 (0.0008) [2023-10-12 20:08:21,734][44959] Updated weights for policy 1, policy_version 4760 (0.0008) [2023-10-12 20:08:21,776][44518] Saving new best policy, reward=176.720! [2023-10-12 20:08:25,813][44958] Updated weights for policy 0, policy_version 4740 (0.0007) [2023-10-12 20:08:26,141][44959] Updated weights for policy 1, policy_version 4770 (0.0008) [2023-10-12 20:08:26,185][44958] Updated weights for policy 0, policy_version 4750 (0.0009) [2023-10-12 20:08:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 9732096. Throughput: 0: 1634.2, 1: 1642.0. Samples: 2448768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:08:26,443][43579] Avg episode reward: [(0, '179.450'), (1, '188.950')] [2023-10-12 20:08:26,515][44959] Updated weights for policy 1, policy_version 4780 (0.0008) [2023-10-12 20:08:26,548][44958] Updated weights for policy 0, policy_version 4760 (0.0008) [2023-10-12 20:08:26,843][44518] Saving new best policy, reward=179.450! [2023-10-12 20:08:26,883][44959] Updated weights for policy 1, policy_version 4790 (0.0007) [2023-10-12 20:08:27,254][44959] Updated weights for policy 1, policy_version 4800 (0.0009) [2023-10-12 20:08:30,780][44958] Updated weights for policy 0, policy_version 4770 (0.0008) [2023-10-12 20:08:31,147][44958] Updated weights for policy 0, policy_version 4780 (0.0007) [2023-10-12 20:08:31,290][44959] Updated weights for policy 1, policy_version 4810 (0.0008) [2023-10-12 20:08:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9797632. Throughput: 0: 1636.6, 1: 1644.0. Samples: 2468346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:08:31,443][43579] Avg episode reward: [(0, '182.160'), (1, '180.220')] [2023-10-12 20:08:31,515][44958] Updated weights for policy 0, policy_version 4790 (0.0009) [2023-10-12 20:08:31,653][44959] Updated weights for policy 1, policy_version 4820 (0.0008) [2023-10-12 20:08:31,888][44958] Updated weights for policy 0, policy_version 4800 (0.0008) [2023-10-12 20:08:31,888][44518] Saving new best policy, reward=182.160! [2023-10-12 20:08:32,032][44959] Updated weights for policy 1, policy_version 4830 (0.0010) [2023-10-12 20:08:35,743][44958] Updated weights for policy 0, policy_version 4810 (0.0008) [2023-10-12 20:08:36,120][44958] Updated weights for policy 0, policy_version 4820 (0.0008) [2023-10-12 20:08:36,252][44959] Updated weights for policy 1, policy_version 4840 (0.0008) [2023-10-12 20:08:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 9863168. Throughput: 0: 1638.6, 1: 1647.9. Samples: 2477890. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 20:08:36,443][43579] Avg episode reward: [(0, '181.660'), (1, '182.960')] [2023-10-12 20:08:36,494][44958] Updated weights for policy 0, policy_version 4830 (0.0009) [2023-10-12 20:08:36,628][44959] Updated weights for policy 1, policy_version 4850 (0.0009) [2023-10-12 20:08:36,993][44959] Updated weights for policy 1, policy_version 4860 (0.0008) [2023-10-12 20:08:40,661][44958] Updated weights for policy 0, policy_version 4840 (0.0008) [2023-10-12 20:08:41,026][44958] Updated weights for policy 0, policy_version 4850 (0.0008) [2023-10-12 20:08:41,132][44959] Updated weights for policy 1, policy_version 4870 (0.0007) [2023-10-12 20:08:41,406][44958] Updated weights for policy 0, policy_version 4860 (0.0007) [2023-10-12 20:08:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9928704. Throughput: 0: 1635.3, 1: 1646.5. Samples: 2498134. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 20:08:41,443][43579] Avg episode reward: [(0, '182.340'), (1, '180.990')] [2023-10-12 20:08:41,503][44959] Updated weights for policy 1, policy_version 4880 (0.0008) [2023-10-12 20:08:41,545][44518] Saving new best policy, reward=182.340! [2023-10-12 20:08:41,868][44959] Updated weights for policy 1, policy_version 4890 (0.0009) [2023-10-12 20:08:45,676][44958] Updated weights for policy 0, policy_version 4870 (0.0008) [2023-10-12 20:08:46,054][44958] Updated weights for policy 0, policy_version 4880 (0.0009) [2023-10-12 20:08:46,080][44959] Updated weights for policy 1, policy_version 4900 (0.0007) [2023-10-12 20:08:46,419][44958] Updated weights for policy 0, policy_version 4890 (0.0007) [2023-10-12 20:08:46,443][44959] Updated weights for policy 1, policy_version 4910 (0.0007) [2023-10-12 20:08:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9994240. Throughput: 0: 1637.5, 1: 1641.3. Samples: 2517568. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-10-12 20:08:46,444][43579] Avg episode reward: [(0, '175.680'), (1, '188.580')] [2023-10-12 20:08:46,818][44959] Updated weights for policy 1, policy_version 4920 (0.0009) [2023-10-12 20:08:50,562][44958] Updated weights for policy 0, policy_version 4900 (0.0008) [2023-10-12 20:08:50,923][44958] Updated weights for policy 0, policy_version 4910 (0.0009) [2023-10-12 20:08:50,937][44959] Updated weights for policy 1, policy_version 4930 (0.0008) [2023-10-12 20:08:51,304][44958] Updated weights for policy 0, policy_version 4920 (0.0009) [2023-10-12 20:08:51,309][44959] Updated weights for policy 1, policy_version 4940 (0.0009) [2023-10-12 20:08:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10059776. Throughput: 0: 1640.5, 1: 1642.6. Samples: 2527166. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-10-12 20:08:51,444][43579] Avg episode reward: [(0, '176.370'), (1, '188.940')] [2023-10-12 20:08:51,683][44959] Updated weights for policy 1, policy_version 4950 (0.0007) [2023-10-12 20:08:52,053][44959] Updated weights for policy 1, policy_version 4960 (0.0008) [2023-10-12 20:08:55,471][44958] Updated weights for policy 0, policy_version 4930 (0.0010) [2023-10-12 20:08:55,838][44958] Updated weights for policy 0, policy_version 4940 (0.0008) [2023-10-12 20:08:56,209][44958] Updated weights for policy 0, policy_version 4950 (0.0009) [2023-10-12 20:08:56,230][44959] Updated weights for policy 1, policy_version 4970 (0.0008) [2023-10-12 20:08:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10125312. Throughput: 0: 1636.6, 1: 1640.6. Samples: 2547200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:08:56,443][43579] Avg episode reward: [(0, '178.890'), (1, '192.890')] [2023-10-12 20:08:56,577][44958] Updated weights for policy 0, policy_version 4960 (0.0008) [2023-10-12 20:08:56,587][44959] Updated weights for policy 1, policy_version 4980 (0.0009) [2023-10-12 20:08:56,960][44959] Updated weights for policy 1, policy_version 4990 (0.0008) [2023-10-12 20:08:57,030][44583] Saving new best policy, reward=192.890! [2023-10-12 20:09:00,892][44958] Updated weights for policy 0, policy_version 4970 (0.0009) [2023-10-12 20:09:01,236][44959] Updated weights for policy 1, policy_version 5000 (0.0008) [2023-10-12 20:09:01,254][44958] Updated weights for policy 0, policy_version 4980 (0.0009) [2023-10-12 20:09:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10190848. Throughput: 0: 1639.8, 1: 1638.0. Samples: 2566606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:09:01,443][43579] Avg episode reward: [(0, '182.920'), (1, '200.900')] [2023-10-12 20:09:01,602][44959] Updated weights for policy 1, policy_version 5010 (0.0009) [2023-10-12 20:09:01,630][44958] Updated weights for policy 0, policy_version 4990 (0.0009) [2023-10-12 20:09:01,697][44518] Saving new best policy, reward=182.920! [2023-10-12 20:09:01,969][44959] Updated weights for policy 1, policy_version 5020 (0.0007) [2023-10-12 20:09:02,116][44583] Saving new best policy, reward=200.900! [2023-10-12 20:09:05,883][44958] Updated weights for policy 0, policy_version 5000 (0.0009) [2023-10-12 20:09:06,095][44959] Updated weights for policy 1, policy_version 5030 (0.0007) [2023-10-12 20:09:06,256][44958] Updated weights for policy 0, policy_version 5010 (0.0009) [2023-10-12 20:09:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10256384. Throughput: 0: 1646.0, 1: 1636.3. Samples: 2576210. Policy #0 lag: (min: 8.0, avg: 31.7, max: 40.0) [2023-10-12 20:09:06,443][43579] Avg episode reward: [(0, '186.190'), (1, '200.110')] [2023-10-12 20:09:06,455][44959] Updated weights for policy 1, policy_version 5040 (0.0008) [2023-10-12 20:09:06,625][44958] Updated weights for policy 0, policy_version 5020 (0.0008) [2023-10-12 20:09:06,771][44518] Saving new best policy, reward=186.190! [2023-10-12 20:09:06,834][44959] Updated weights for policy 1, policy_version 5050 (0.0009) [2023-10-12 20:09:10,980][44958] Updated weights for policy 0, policy_version 5030 (0.0010) [2023-10-12 20:09:11,124][44959] Updated weights for policy 1, policy_version 5060 (0.0008) [2023-10-12 20:09:11,349][44958] Updated weights for policy 0, policy_version 5040 (0.0009) [2023-10-12 20:09:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10321920. Throughput: 0: 1640.4, 1: 1641.2. Samples: 2596444. Policy #0 lag: (min: 8.0, avg: 31.7, max: 40.0) [2023-10-12 20:09:11,443][43579] Avg episode reward: [(0, '189.390'), (1, '200.320')] [2023-10-12 20:09:11,486][44959] Updated weights for policy 1, policy_version 5070 (0.0008) [2023-10-12 20:09:11,719][44958] Updated weights for policy 0, policy_version 5050 (0.0008) [2023-10-12 20:09:11,856][44959] Updated weights for policy 1, policy_version 5080 (0.0008) [2023-10-12 20:09:11,941][44518] Saving new best policy, reward=189.390! [2023-10-12 20:09:15,747][44958] Updated weights for policy 0, policy_version 5060 (0.0010) [2023-10-12 20:09:15,956][44959] Updated weights for policy 1, policy_version 5090 (0.0007) [2023-10-12 20:09:16,110][44958] Updated weights for policy 0, policy_version 5070 (0.0008) [2023-10-12 20:09:16,323][44959] Updated weights for policy 1, policy_version 5100 (0.0009) [2023-10-12 20:09:16,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10387456. Throughput: 0: 1642.5, 1: 1634.8. Samples: 2615824. Policy #0 lag: (min: 27.0, avg: 30.8, max: 59.0) [2023-10-12 20:09:16,443][43579] Avg episode reward: [(0, '197.230'), (1, '197.360')] [2023-10-12 20:09:16,483][44958] Updated weights for policy 0, policy_version 5080 (0.0010) [2023-10-12 20:09:16,695][44959] Updated weights for policy 1, policy_version 5110 (0.0009) [2023-10-12 20:09:16,778][44518] Saving new best policy, reward=197.230! [2023-10-12 20:09:17,061][44959] Updated weights for policy 1, policy_version 5120 (0.0010) [2023-10-12 20:09:20,803][44958] Updated weights for policy 0, policy_version 5090 (0.0009) [2023-10-12 20:09:21,172][44958] Updated weights for policy 0, policy_version 5100 (0.0009) [2023-10-12 20:09:21,401][44959] Updated weights for policy 1, policy_version 5130 (0.0008) [2023-10-12 20:09:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10452992. Throughput: 0: 1637.9, 1: 1636.7. Samples: 2625248. Policy #0 lag: (min: 27.0, avg: 30.8, max: 59.0) [2023-10-12 20:09:21,444][43579] Avg episode reward: [(0, '199.600'), (1, '200.910')] [2023-10-12 20:09:21,552][44958] Updated weights for policy 0, policy_version 5110 (0.0008) [2023-10-12 20:09:21,780][44959] Updated weights for policy 1, policy_version 5140 (0.0009) [2023-10-12 20:09:21,917][44958] Updated weights for policy 0, policy_version 5120 (0.0008) [2023-10-12 20:09:21,917][44518] Saving new best policy, reward=199.600! [2023-10-12 20:09:22,139][44959] Updated weights for policy 1, policy_version 5150 (0.0007) [2023-10-12 20:09:22,214][44583] Saving new best policy, reward=200.910! [2023-10-12 20:09:26,241][44958] Updated weights for policy 0, policy_version 5130 (0.0009) [2023-10-12 20:09:26,400][44959] Updated weights for policy 1, policy_version 5160 (0.0008) [2023-10-12 20:09:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 10518528. Throughput: 0: 1629.6, 1: 1633.5. Samples: 2644976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:09:26,444][43579] Avg episode reward: [(0, '194.990'), (1, '202.890')] [2023-10-12 20:09:26,619][44958] Updated weights for policy 0, policy_version 5140 (0.0009) [2023-10-12 20:09:26,760][44959] Updated weights for policy 1, policy_version 5170 (0.0008) [2023-10-12 20:09:26,997][44958] Updated weights for policy 0, policy_version 5150 (0.0008) [2023-10-12 20:09:27,124][44959] Updated weights for policy 1, policy_version 5180 (0.0010) [2023-10-12 20:09:27,273][44583] Saving new best policy, reward=202.890! [2023-10-12 20:09:30,997][44958] Updated weights for policy 0, policy_version 5160 (0.0009) [2023-10-12 20:09:31,073][44959] Updated weights for policy 1, policy_version 5190 (0.0009) [2023-10-12 20:09:31,365][44958] Updated weights for policy 0, policy_version 5170 (0.0009) [2023-10-12 20:09:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10584064. Throughput: 0: 1630.4, 1: 1638.1. Samples: 2664648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:09:31,443][43579] Avg episode reward: [(0, '193.660'), (1, '202.440')] [2023-10-12 20:09:31,444][44959] Updated weights for policy 1, policy_version 5200 (0.0008) [2023-10-12 20:09:31,739][44958] Updated weights for policy 0, policy_version 5180 (0.0008) [2023-10-12 20:09:31,805][44959] Updated weights for policy 1, policy_version 5210 (0.0009) [2023-10-12 20:09:35,818][44959] Updated weights for policy 1, policy_version 5220 (0.0010) [2023-10-12 20:09:36,060][44958] Updated weights for policy 0, policy_version 5190 (0.0008) [2023-10-12 20:09:36,184][44959] Updated weights for policy 1, policy_version 5230 (0.0010) [2023-10-12 20:09:36,438][44958] Updated weights for policy 0, policy_version 5200 (0.0007) [2023-10-12 20:09:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 10649600. Throughput: 0: 1625.6, 1: 1640.9. Samples: 2674162. Policy #0 lag: (min: 26.0, avg: 28.0, max: 58.0) [2023-10-12 20:09:36,443][43579] Avg episode reward: [(0, '189.120'), (1, '206.280')] [2023-10-12 20:09:36,551][44959] Updated weights for policy 1, policy_version 5240 (0.0007) [2023-10-12 20:09:36,807][44958] Updated weights for policy 0, policy_version 5210 (0.0008) [2023-10-12 20:09:36,849][44583] Saving new best policy, reward=206.280! [2023-10-12 20:09:40,573][44959] Updated weights for policy 1, policy_version 5250 (0.0009) [2023-10-12 20:09:40,835][44958] Updated weights for policy 0, policy_version 5220 (0.0008) [2023-10-12 20:09:40,953][44959] Updated weights for policy 1, policy_version 5260 (0.0008) [2023-10-12 20:09:41,210][44958] Updated weights for policy 0, policy_version 5230 (0.0008) [2023-10-12 20:09:41,325][44959] Updated weights for policy 1, policy_version 5270 (0.0009) [2023-10-12 20:09:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10715136. Throughput: 0: 1631.1, 1: 1646.4. Samples: 2694684. Policy #0 lag: (min: 26.0, avg: 28.0, max: 58.0) [2023-10-12 20:09:41,443][43579] Avg episode reward: [(0, '189.710'), (1, '202.750')] [2023-10-12 20:09:41,589][44958] Updated weights for policy 0, policy_version 5240 (0.0010) [2023-10-12 20:09:41,691][44959] Updated weights for policy 1, policy_version 5280 (0.0009) [2023-10-12 20:09:45,616][44958] Updated weights for policy 0, policy_version 5250 (0.0009) [2023-10-12 20:09:45,992][44958] Updated weights for policy 0, policy_version 5260 (0.0008) [2023-10-12 20:09:46,037][44959] Updated weights for policy 1, policy_version 5290 (0.0008) [2023-10-12 20:09:46,361][44958] Updated weights for policy 0, policy_version 5270 (0.0009) [2023-10-12 20:09:46,407][44959] Updated weights for policy 1, policy_version 5300 (0.0010) [2023-10-12 20:09:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10780672. Throughput: 0: 1633.3, 1: 1641.4. Samples: 2713968. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-12 20:09:46,443][43579] Avg episode reward: [(0, '190.750'), (1, '205.620')] [2023-10-12 20:09:46,726][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000005280_5406720.pth... [2023-10-12 20:09:46,730][44958] Updated weights for policy 0, policy_version 5280 (0.0007) [2023-10-12 20:09:46,755][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000003744_3833856.pth [2023-10-12 20:09:46,781][44959] Updated weights for policy 1, policy_version 5310 (0.0007) [2023-10-12 20:09:46,851][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000005312_5439488.pth... [2023-10-12 20:09:46,883][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000003744_3833856.pth [2023-10-12 20:09:50,799][44959] Updated weights for policy 1, policy_version 5320 (0.0009) [2023-10-12 20:09:51,126][44958] Updated weights for policy 0, policy_version 5290 (0.0008) [2023-10-12 20:09:51,162][44959] Updated weights for policy 1, policy_version 5330 (0.0009) [2023-10-12 20:09:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10846208. Throughput: 0: 1629.9, 1: 1648.0. Samples: 2723716. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-12 20:09:51,443][43579] Avg episode reward: [(0, '193.950'), (1, '208.280')] [2023-10-12 20:09:51,500][44958] Updated weights for policy 0, policy_version 5300 (0.0008) [2023-10-12 20:09:51,527][44959] Updated weights for policy 1, policy_version 5340 (0.0008) [2023-10-12 20:09:51,665][44583] Saving new best policy, reward=208.280! [2023-10-12 20:09:51,874][44958] Updated weights for policy 0, policy_version 5310 (0.0010) [2023-10-12 20:09:55,628][44959] Updated weights for policy 1, policy_version 5350 (0.0010) [2023-10-12 20:09:55,999][44959] Updated weights for policy 1, policy_version 5360 (0.0010) [2023-10-12 20:09:56,097][44958] Updated weights for policy 0, policy_version 5320 (0.0011) [2023-10-12 20:09:56,363][44959] Updated weights for policy 1, policy_version 5370 (0.0007) [2023-10-12 20:09:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10911744. Throughput: 0: 1631.9, 1: 1646.3. Samples: 2743962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:09:56,443][43579] Avg episode reward: [(0, '196.200'), (1, '204.210')] [2023-10-12 20:09:56,457][44958] Updated weights for policy 0, policy_version 5330 (0.0008) [2023-10-12 20:09:56,828][44958] Updated weights for policy 0, policy_version 5340 (0.0009) [2023-10-12 20:10:00,644][44959] Updated weights for policy 1, policy_version 5380 (0.0007) [2023-10-12 20:10:00,941][44958] Updated weights for policy 0, policy_version 5350 (0.0007) [2023-10-12 20:10:01,008][44959] Updated weights for policy 1, policy_version 5390 (0.0008) [2023-10-12 20:10:01,315][44958] Updated weights for policy 0, policy_version 5360 (0.0009) [2023-10-12 20:10:01,380][44959] Updated weights for policy 1, policy_version 5400 (0.0009) [2023-10-12 20:10:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10977280. Throughput: 0: 1638.1, 1: 1635.5. Samples: 2763134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:10:01,443][43579] Avg episode reward: [(0, '202.500'), (1, '209.100')] [2023-10-12 20:10:01,672][44583] Saving new best policy, reward=209.100! [2023-10-12 20:10:01,680][44958] Updated weights for policy 0, policy_version 5370 (0.0009) [2023-10-12 20:10:01,902][44518] Saving new best policy, reward=202.500! [2023-10-12 20:10:05,691][44959] Updated weights for policy 1, policy_version 5410 (0.0008) [2023-10-12 20:10:05,765][44958] Updated weights for policy 0, policy_version 5380 (0.0008) [2023-10-12 20:10:06,094][44959] Updated weights for policy 1, policy_version 5420 (0.0008) [2023-10-12 20:10:06,132][44958] Updated weights for policy 0, policy_version 5390 (0.0007) [2023-10-12 20:10:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11042816. Throughput: 0: 1639.2, 1: 1640.7. Samples: 2772842. Policy #0 lag: (min: 4.0, avg: 10.1, max: 36.0) [2023-10-12 20:10:06,443][43579] Avg episode reward: [(0, '208.160'), (1, '212.970')] [2023-10-12 20:10:06,458][44959] Updated weights for policy 1, policy_version 5430 (0.0010) [2023-10-12 20:10:06,505][44958] Updated weights for policy 0, policy_version 5400 (0.0008) [2023-10-12 20:10:06,795][44518] Saving new best policy, reward=208.160! [2023-10-12 20:10:06,825][44583] Saving new best policy, reward=212.970! [2023-10-12 20:10:06,829][44959] Updated weights for policy 1, policy_version 5440 (0.0009) [2023-10-12 20:10:10,932][44958] Updated weights for policy 0, policy_version 5410 (0.0009) [2023-10-12 20:10:10,989][44959] Updated weights for policy 1, policy_version 5450 (0.0009) [2023-10-12 20:10:11,292][44958] Updated weights for policy 0, policy_version 5420 (0.0009) [2023-10-12 20:10:11,362][44959] Updated weights for policy 1, policy_version 5460 (0.0008) [2023-10-12 20:10:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11108352. Throughput: 0: 1646.4, 1: 1641.4. Samples: 2792926. Policy #0 lag: (min: 4.0, avg: 10.1, max: 36.0) [2023-10-12 20:10:11,444][43579] Avg episode reward: [(0, '209.950'), (1, '211.430')] [2023-10-12 20:10:11,659][44958] Updated weights for policy 0, policy_version 5430 (0.0008) [2023-10-12 20:10:11,729][44959] Updated weights for policy 1, policy_version 5470 (0.0008) [2023-10-12 20:10:12,028][44518] Saving new best policy, reward=209.950! [2023-10-12 20:10:12,032][44958] Updated weights for policy 0, policy_version 5440 (0.0008) [2023-10-12 20:10:16,114][44959] Updated weights for policy 1, policy_version 5480 (0.0008) [2023-10-12 20:10:16,161][44958] Updated weights for policy 0, policy_version 5450 (0.0008) [2023-10-12 20:10:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 11173888. Throughput: 0: 1643.9, 1: 1636.2. Samples: 2812252. Policy #0 lag: (min: 2.0, avg: 6.8, max: 34.0) [2023-10-12 20:10:16,443][43579] Avg episode reward: [(0, '210.660'), (1, '207.540')] [2023-10-12 20:10:16,476][44959] Updated weights for policy 1, policy_version 5490 (0.0008) [2023-10-12 20:10:16,525][44958] Updated weights for policy 0, policy_version 5460 (0.0008) [2023-10-12 20:10:16,850][44959] Updated weights for policy 1, policy_version 5500 (0.0008) [2023-10-12 20:10:16,894][44958] Updated weights for policy 0, policy_version 5470 (0.0011) [2023-10-12 20:10:16,966][44518] Saving new best policy, reward=210.660! [2023-10-12 20:10:21,076][44958] Updated weights for policy 0, policy_version 5480 (0.0009) [2023-10-12 20:10:21,178][44959] Updated weights for policy 1, policy_version 5510 (0.0008) [2023-10-12 20:10:21,439][44958] Updated weights for policy 0, policy_version 5490 (0.0009) [2023-10-12 20:10:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11239424. Throughput: 0: 1644.5, 1: 1637.1. Samples: 2821836. Policy #0 lag: (min: 2.0, avg: 6.8, max: 34.0) [2023-10-12 20:10:21,444][43579] Avg episode reward: [(0, '208.950'), (1, '207.400')] [2023-10-12 20:10:21,547][44959] Updated weights for policy 1, policy_version 5520 (0.0007) [2023-10-12 20:10:21,812][44958] Updated weights for policy 0, policy_version 5500 (0.0008) [2023-10-12 20:10:21,910][44959] Updated weights for policy 1, policy_version 5530 (0.0008) [2023-10-12 20:10:25,874][44959] Updated weights for policy 1, policy_version 5540 (0.0009) [2023-10-12 20:10:26,009][44958] Updated weights for policy 0, policy_version 5510 (0.0009) [2023-10-12 20:10:26,247][44959] Updated weights for policy 1, policy_version 5550 (0.0008) [2023-10-12 20:10:26,376][44958] Updated weights for policy 0, policy_version 5520 (0.0009) [2023-10-12 20:10:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11304960. Throughput: 0: 1638.6, 1: 1637.1. Samples: 2842090. Policy #0 lag: (min: 17.0, avg: 29.3, max: 49.0) [2023-10-12 20:10:26,443][43579] Avg episode reward: [(0, '210.420'), (1, '207.980')] [2023-10-12 20:10:26,621][44959] Updated weights for policy 1, policy_version 5560 (0.0008) [2023-10-12 20:10:26,757][44958] Updated weights for policy 0, policy_version 5530 (0.0007) [2023-10-12 20:10:30,928][44958] Updated weights for policy 0, policy_version 5540 (0.0007) [2023-10-12 20:10:30,936][44959] Updated weights for policy 1, policy_version 5570 (0.0009) [2023-10-12 20:10:31,296][44958] Updated weights for policy 0, policy_version 5550 (0.0007) [2023-10-12 20:10:31,310][44959] Updated weights for policy 1, policy_version 5580 (0.0009) [2023-10-12 20:10:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11370496. Throughput: 0: 1637.1, 1: 1645.6. Samples: 2861688. Policy #0 lag: (min: 17.0, avg: 29.3, max: 49.0) [2023-10-12 20:10:31,443][43579] Avg episode reward: [(0, '212.200'), (1, '213.940')] [2023-10-12 20:10:31,673][44958] Updated weights for policy 0, policy_version 5560 (0.0007) [2023-10-12 20:10:31,678][44959] Updated weights for policy 1, policy_version 5590 (0.0008) [2023-10-12 20:10:31,964][44518] Saving new best policy, reward=212.200! [2023-10-12 20:10:32,040][44583] Saving new best policy, reward=213.940! [2023-10-12 20:10:32,044][44959] Updated weights for policy 1, policy_version 5600 (0.0008) [2023-10-12 20:10:35,901][44958] Updated weights for policy 0, policy_version 5570 (0.0009) [2023-10-12 20:10:36,279][44958] Updated weights for policy 0, policy_version 5580 (0.0009) [2023-10-12 20:10:36,313][44959] Updated weights for policy 1, policy_version 5610 (0.0007) [2023-10-12 20:10:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11436032. Throughput: 0: 1637.2, 1: 1639.3. Samples: 2871160. Policy #0 lag: (min: 17.0, avg: 24.7, max: 49.0) [2023-10-12 20:10:36,443][43579] Avg episode reward: [(0, '211.970'), (1, '213.010')] [2023-10-12 20:10:36,645][44958] Updated weights for policy 0, policy_version 5590 (0.0007) [2023-10-12 20:10:36,688][44959] Updated weights for policy 1, policy_version 5620 (0.0007) [2023-10-12 20:10:37,017][44958] Updated weights for policy 0, policy_version 5600 (0.0007) [2023-10-12 20:10:37,051][44959] Updated weights for policy 1, policy_version 5630 (0.0008) [2023-10-12 20:10:40,913][44958] Updated weights for policy 0, policy_version 5610 (0.0008) [2023-10-12 20:10:41,288][44959] Updated weights for policy 1, policy_version 5640 (0.0009) [2023-10-12 20:10:41,289][44958] Updated weights for policy 0, policy_version 5620 (0.0008) [2023-10-12 20:10:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 11501568. Throughput: 0: 1636.8, 1: 1639.2. Samples: 2891384. Policy #0 lag: (min: 17.0, avg: 24.7, max: 49.0) [2023-10-12 20:10:41,444][43579] Avg episode reward: [(0, '215.260'), (1, '210.490')] [2023-10-12 20:10:41,663][44959] Updated weights for policy 1, policy_version 5650 (0.0009) [2023-10-12 20:10:41,663][44958] Updated weights for policy 0, policy_version 5630 (0.0009) [2023-10-12 20:10:41,736][44518] Saving new best policy, reward=215.260! [2023-10-12 20:10:42,035][44959] Updated weights for policy 1, policy_version 5660 (0.0010) [2023-10-12 20:10:45,807][44958] Updated weights for policy 0, policy_version 5640 (0.0007) [2023-10-12 20:10:46,089][44959] Updated weights for policy 1, policy_version 5670 (0.0008) [2023-10-12 20:10:46,175][44958] Updated weights for policy 0, policy_version 5650 (0.0008) [2023-10-12 20:10:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11567104. Throughput: 0: 1635.9, 1: 1652.1. Samples: 2911096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:10:46,443][43579] Avg episode reward: [(0, '212.250'), (1, '215.540')] [2023-10-12 20:10:46,466][44959] Updated weights for policy 1, policy_version 5680 (0.0007) [2023-10-12 20:10:46,544][44958] Updated weights for policy 0, policy_version 5660 (0.0009) [2023-10-12 20:10:46,834][44959] Updated weights for policy 1, policy_version 5690 (0.0009) [2023-10-12 20:10:47,051][44583] Saving new best policy, reward=215.540! [2023-10-12 20:10:50,741][44958] Updated weights for policy 0, policy_version 5670 (0.0007) [2023-10-12 20:10:51,109][44958] Updated weights for policy 0, policy_version 5680 (0.0007) [2023-10-12 20:10:51,128][44959] Updated weights for policy 1, policy_version 5700 (0.0009) [2023-10-12 20:10:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11632640. Throughput: 0: 1639.3, 1: 1646.8. Samples: 2920718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:10:51,443][43579] Avg episode reward: [(0, '215.860'), (1, '216.970')] [2023-10-12 20:10:51,479][44958] Updated weights for policy 0, policy_version 5690 (0.0008) [2023-10-12 20:10:51,520][44959] Updated weights for policy 1, policy_version 5710 (0.0009) [2023-10-12 20:10:51,700][44518] Saving new best policy, reward=215.860! [2023-10-12 20:10:51,886][44959] Updated weights for policy 1, policy_version 5720 (0.0008) [2023-10-12 20:10:52,178][44583] Saving new best policy, reward=216.970! [2023-10-12 20:10:55,593][44958] Updated weights for policy 0, policy_version 5700 (0.0008) [2023-10-12 20:10:55,964][44958] Updated weights for policy 0, policy_version 5710 (0.0008) [2023-10-12 20:10:55,982][44959] Updated weights for policy 1, policy_version 5730 (0.0008) [2023-10-12 20:10:56,340][44958] Updated weights for policy 0, policy_version 5720 (0.0009) [2023-10-12 20:10:56,342][44959] Updated weights for policy 1, policy_version 5740 (0.0008) [2023-10-12 20:10:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11698176. Throughput: 0: 1640.1, 1: 1648.6. Samples: 2940916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:10:56,443][43579] Avg episode reward: [(0, '217.590'), (1, '220.180')] [2023-10-12 20:10:56,632][44518] Saving new best policy, reward=217.590! [2023-10-12 20:10:56,703][44959] Updated weights for policy 1, policy_version 5750 (0.0009) [2023-10-12 20:10:57,069][44583] Saving new best policy, reward=220.180! [2023-10-12 20:10:57,072][44959] Updated weights for policy 1, policy_version 5760 (0.0009) [2023-10-12 20:11:00,435][44958] Updated weights for policy 0, policy_version 5730 (0.0009) [2023-10-12 20:11:00,806][44958] Updated weights for policy 0, policy_version 5740 (0.0010) [2023-10-12 20:11:01,081][44959] Updated weights for policy 1, policy_version 5770 (0.0009) [2023-10-12 20:11:01,187][44958] Updated weights for policy 0, policy_version 5750 (0.0008) [2023-10-12 20:11:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11763712. Throughput: 0: 1637.2, 1: 1647.3. Samples: 2960056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:11:01,443][43579] Avg episode reward: [(0, '211.840'), (1, '219.620')] [2023-10-12 20:11:01,461][44959] Updated weights for policy 1, policy_version 5780 (0.0008) [2023-10-12 20:11:01,557][44958] Updated weights for policy 0, policy_version 5760 (0.0010) [2023-10-12 20:11:01,820][44959] Updated weights for policy 1, policy_version 5790 (0.0008) [2023-10-12 20:11:05,817][44958] Updated weights for policy 0, policy_version 5770 (0.0009) [2023-10-12 20:11:05,971][44959] Updated weights for policy 1, policy_version 5800 (0.0009) [2023-10-12 20:11:06,179][44958] Updated weights for policy 0, policy_version 5780 (0.0009) [2023-10-12 20:11:06,336][44959] Updated weights for policy 1, policy_version 5810 (0.0008) [2023-10-12 20:11:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11829248. Throughput: 0: 1646.4, 1: 1652.5. Samples: 2970284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:11:06,444][43579] Avg episode reward: [(0, '207.790'), (1, '226.920')] [2023-10-12 20:11:06,559][44958] Updated weights for policy 0, policy_version 5790 (0.0008) [2023-10-12 20:11:06,707][44959] Updated weights for policy 1, policy_version 5820 (0.0009) [2023-10-12 20:11:06,856][44583] Saving new best policy, reward=226.920! [2023-10-12 20:11:10,689][44958] Updated weights for policy 0, policy_version 5800 (0.0008) [2023-10-12 20:11:10,735][44959] Updated weights for policy 1, policy_version 5830 (0.0009) [2023-10-12 20:11:11,064][44958] Updated weights for policy 0, policy_version 5810 (0.0008) [2023-10-12 20:11:11,095][44959] Updated weights for policy 1, policy_version 5840 (0.0007) [2023-10-12 20:11:11,434][44958] Updated weights for policy 0, policy_version 5820 (0.0008) [2023-10-12 20:11:11,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 11894784. Throughput: 0: 1647.4, 1: 1650.7. Samples: 2990506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:11:11,443][43579] Avg episode reward: [(0, '209.080'), (1, '229.850')] [2023-10-12 20:11:11,464][44959] Updated weights for policy 1, policy_version 5850 (0.0009) [2023-10-12 20:11:11,685][44583] Saving new best policy, reward=229.850! [2023-10-12 20:11:15,477][44958] Updated weights for policy 0, policy_version 5830 (0.0009) [2023-10-12 20:11:15,688][44959] Updated weights for policy 1, policy_version 5860 (0.0008) [2023-10-12 20:11:15,845][44958] Updated weights for policy 0, policy_version 5840 (0.0007) [2023-10-12 20:11:16,057][44959] Updated weights for policy 1, policy_version 5870 (0.0007) [2023-10-12 20:11:16,219][44958] Updated weights for policy 0, policy_version 5850 (0.0007) [2023-10-12 20:11:16,417][44959] Updated weights for policy 1, policy_version 5880 (0.0008) [2023-10-12 20:11:16,442][43579] Fps is (10 sec: 16384.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 11993088. Throughput: 0: 1643.9, 1: 1638.6. Samples: 3009400. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 20:11:16,443][43579] Avg episode reward: [(0, '209.440'), (1, '225.450')] [2023-10-12 20:11:20,460][44958] Updated weights for policy 0, policy_version 5860 (0.0009) [2023-10-12 20:11:20,650][44959] Updated weights for policy 1, policy_version 5890 (0.0008) [2023-10-12 20:11:20,834][44958] Updated weights for policy 0, policy_version 5870 (0.0010) [2023-10-12 20:11:21,019][44959] Updated weights for policy 1, policy_version 5900 (0.0007) [2023-10-12 20:11:21,201][44958] Updated weights for policy 0, policy_version 5880 (0.0007) [2023-10-12 20:11:21,380][44959] Updated weights for policy 1, policy_version 5910 (0.0007) [2023-10-12 20:11:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12025856. Throughput: 0: 1651.9, 1: 1646.1. Samples: 3019570. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 20:11:21,443][43579] Avg episode reward: [(0, '208.860'), (1, '225.260')] [2023-10-12 20:11:21,753][44959] Updated weights for policy 1, policy_version 5920 (0.0008) [2023-10-12 20:11:25,696][44958] Updated weights for policy 0, policy_version 5890 (0.0009) [2023-10-12 20:11:26,031][44959] Updated weights for policy 1, policy_version 5930 (0.0008) [2023-10-12 20:11:26,114][44958] Updated weights for policy 0, policy_version 5900 (0.0008) [2023-10-12 20:11:26,398][44959] Updated weights for policy 1, policy_version 5940 (0.0008) [2023-10-12 20:11:26,442][43579] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12091392. Throughput: 0: 1642.5, 1: 1648.6. Samples: 3039486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:11:26,443][43579] Avg episode reward: [(0, '215.080'), (1, '220.860')] [2023-10-12 20:11:26,483][44958] Updated weights for policy 0, policy_version 5910 (0.0009) [2023-10-12 20:11:26,773][44959] Updated weights for policy 1, policy_version 5950 (0.0007) [2023-10-12 20:11:26,854][44958] Updated weights for policy 0, policy_version 5920 (0.0010) [2023-10-12 20:11:30,759][44959] Updated weights for policy 1, policy_version 5960 (0.0009) [2023-10-12 20:11:30,911][44958] Updated weights for policy 0, policy_version 5930 (0.0009) [2023-10-12 20:11:31,129][44959] Updated weights for policy 1, policy_version 5970 (0.0008) [2023-10-12 20:11:31,284][44958] Updated weights for policy 0, policy_version 5940 (0.0008) [2023-10-12 20:11:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 12156928. Throughput: 0: 1637.9, 1: 1638.4. Samples: 3058530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:11:31,443][43579] Avg episode reward: [(0, '214.530'), (1, '219.840')] [2023-10-12 20:11:31,501][44959] Updated weights for policy 1, policy_version 5980 (0.0009) [2023-10-12 20:11:31,651][44958] Updated weights for policy 0, policy_version 5950 (0.0009) [2023-10-12 20:11:35,900][44959] Updated weights for policy 1, policy_version 5990 (0.0007) [2023-10-12 20:11:35,901][44958] Updated weights for policy 0, policy_version 5960 (0.0010) [2023-10-12 20:11:36,270][44958] Updated weights for policy 0, policy_version 5970 (0.0007) [2023-10-12 20:11:36,272][44959] Updated weights for policy 1, policy_version 6000 (0.0008) [2023-10-12 20:11:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 12222464. Throughput: 0: 1635.3, 1: 1646.7. Samples: 3068408. Policy #0 lag: (min: 17.0, avg: 25.2, max: 49.0) [2023-10-12 20:11:36,444][43579] Avg episode reward: [(0, '216.740'), (1, '220.500')] [2023-10-12 20:11:36,637][44958] Updated weights for policy 0, policy_version 5980 (0.0007) [2023-10-12 20:11:36,642][44959] Updated weights for policy 1, policy_version 6010 (0.0007) [2023-10-12 20:11:40,568][44959] Updated weights for policy 1, policy_version 6020 (0.0009) [2023-10-12 20:11:40,940][44959] Updated weights for policy 1, policy_version 6030 (0.0008) [2023-10-12 20:11:41,077][44958] Updated weights for policy 0, policy_version 5990 (0.0009) [2023-10-12 20:11:41,311][44959] Updated weights for policy 1, policy_version 6040 (0.0007) [2023-10-12 20:11:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12288000. Throughput: 0: 1632.7, 1: 1646.7. Samples: 3088488. Policy #0 lag: (min: 17.0, avg: 25.2, max: 49.0) [2023-10-12 20:11:41,443][43579] Avg episode reward: [(0, '218.850'), (1, '215.960')] [2023-10-12 20:11:41,448][44958] Updated weights for policy 0, policy_version 6000 (0.0008) [2023-10-12 20:11:41,824][44958] Updated weights for policy 0, policy_version 6010 (0.0007) [2023-10-12 20:11:42,051][44518] Saving new best policy, reward=218.850! [2023-10-12 20:11:45,599][44959] Updated weights for policy 1, policy_version 6050 (0.0010) [2023-10-12 20:11:45,873][44958] Updated weights for policy 0, policy_version 6020 (0.0008) [2023-10-12 20:11:45,962][44959] Updated weights for policy 1, policy_version 6060 (0.0009) [2023-10-12 20:11:46,248][44958] Updated weights for policy 0, policy_version 6030 (0.0008) [2023-10-12 20:11:46,332][44959] Updated weights for policy 1, policy_version 6070 (0.0008) [2023-10-12 20:11:46,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 12353536. Throughput: 0: 1641.7, 1: 1648.3. Samples: 3108106. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 20:11:46,444][43579] Avg episode reward: [(0, '222.020'), (1, '215.460')] [2023-10-12 20:11:46,626][44958] Updated weights for policy 0, policy_version 6040 (0.0008) [2023-10-12 20:11:46,696][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000006080_6225920.pth... [2023-10-12 20:11:46,697][44959] Updated weights for policy 1, policy_version 6080 (0.0008) [2023-10-12 20:11:46,734][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000004544_4653056.pth [2023-10-12 20:11:46,912][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000006048_6193152.pth... [2023-10-12 20:11:46,953][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000004512_4620288.pth [2023-10-12 20:11:46,958][44518] Saving new best policy, reward=222.020! [2023-10-12 20:11:50,796][44958] Updated weights for policy 0, policy_version 6050 (0.0008) [2023-10-12 20:11:50,813][44959] Updated weights for policy 1, policy_version 6090 (0.0009) [2023-10-12 20:11:51,173][44958] Updated weights for policy 0, policy_version 6060 (0.0008) [2023-10-12 20:11:51,192][44959] Updated weights for policy 1, policy_version 6100 (0.0007) [2023-10-12 20:11:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12419072. Throughput: 0: 1626.5, 1: 1648.7. Samples: 3117666. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 20:11:51,443][43579] Avg episode reward: [(0, '221.290'), (1, '218.460')] [2023-10-12 20:11:51,547][44958] Updated weights for policy 0, policy_version 6070 (0.0007) [2023-10-12 20:11:51,556][44959] Updated weights for policy 1, policy_version 6110 (0.0007) [2023-10-12 20:11:51,921][44958] Updated weights for policy 0, policy_version 6080 (0.0008) [2023-10-12 20:11:55,790][44959] Updated weights for policy 1, policy_version 6120 (0.0008) [2023-10-12 20:11:56,151][44959] Updated weights for policy 1, policy_version 6130 (0.0007) [2023-10-12 20:11:56,155][44958] Updated weights for policy 0, policy_version 6090 (0.0007) [2023-10-12 20:11:56,442][43579] Fps is (10 sec: 13108.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12484608. Throughput: 0: 1633.5, 1: 1642.8. Samples: 3137940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:11:56,443][43579] Avg episode reward: [(0, '221.700'), (1, '218.850')] [2023-10-12 20:11:56,519][44958] Updated weights for policy 0, policy_version 6100 (0.0007) [2023-10-12 20:11:56,525][44959] Updated weights for policy 1, policy_version 6140 (0.0009) [2023-10-12 20:11:56,890][44958] Updated weights for policy 0, policy_version 6110 (0.0008) [2023-10-12 20:12:00,736][44959] Updated weights for policy 1, policy_version 6150 (0.0008) [2023-10-12 20:12:01,105][44959] Updated weights for policy 1, policy_version 6160 (0.0007) [2023-10-12 20:12:01,148][44958] Updated weights for policy 0, policy_version 6120 (0.0009) [2023-10-12 20:12:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12550144. Throughput: 0: 1640.2, 1: 1642.6. Samples: 3157128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:12:01,444][43579] Avg episode reward: [(0, '224.580'), (1, '212.780')] [2023-10-12 20:12:01,481][44959] Updated weights for policy 1, policy_version 6170 (0.0009) [2023-10-12 20:12:01,520][44958] Updated weights for policy 0, policy_version 6130 (0.0007) [2023-10-12 20:12:01,883][44958] Updated weights for policy 0, policy_version 6140 (0.0009) [2023-10-12 20:12:02,034][44518] Saving new best policy, reward=224.580! [2023-10-12 20:12:05,628][44959] Updated weights for policy 1, policy_version 6180 (0.0009) [2023-10-12 20:12:05,994][44959] Updated weights for policy 1, policy_version 6190 (0.0007) [2023-10-12 20:12:06,052][44958] Updated weights for policy 0, policy_version 6150 (0.0007) [2023-10-12 20:12:06,364][44959] Updated weights for policy 1, policy_version 6200 (0.0008) [2023-10-12 20:12:06,437][44958] Updated weights for policy 0, policy_version 6160 (0.0009) [2023-10-12 20:12:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12615680. Throughput: 0: 1628.1, 1: 1644.8. Samples: 3166852. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-12 20:12:06,444][43579] Avg episode reward: [(0, '223.560'), (1, '217.070')] [2023-10-12 20:12:06,812][44958] Updated weights for policy 0, policy_version 6170 (0.0007) [2023-10-12 20:12:10,598][44959] Updated weights for policy 1, policy_version 6210 (0.0007) [2023-10-12 20:12:10,839][44958] Updated weights for policy 0, policy_version 6180 (0.0009) [2023-10-12 20:12:10,977][44959] Updated weights for policy 1, policy_version 6220 (0.0007) [2023-10-12 20:12:11,230][44958] Updated weights for policy 0, policy_version 6190 (0.0007) [2023-10-12 20:12:11,332][44959] Updated weights for policy 1, policy_version 6230 (0.0008) [2023-10-12 20:12:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12681216. Throughput: 0: 1642.1, 1: 1637.4. Samples: 3187066. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-12 20:12:11,443][43579] Avg episode reward: [(0, '216.110'), (1, '223.460')] [2023-10-12 20:12:11,598][44958] Updated weights for policy 0, policy_version 6200 (0.0007) [2023-10-12 20:12:11,702][44959] Updated weights for policy 1, policy_version 6240 (0.0008) [2023-10-12 20:12:15,729][44958] Updated weights for policy 0, policy_version 6210 (0.0008) [2023-10-12 20:12:15,813][44959] Updated weights for policy 1, policy_version 6250 (0.0008) [2023-10-12 20:12:16,103][44958] Updated weights for policy 0, policy_version 6220 (0.0009) [2023-10-12 20:12:16,189][44959] Updated weights for policy 1, policy_version 6260 (0.0009) [2023-10-12 20:12:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 12746752. Throughput: 0: 1638.8, 1: 1640.3. Samples: 3206092. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 20:12:16,443][43579] Avg episode reward: [(0, '217.260'), (1, '221.730')] [2023-10-12 20:12:16,473][44958] Updated weights for policy 0, policy_version 6230 (0.0009) [2023-10-12 20:12:16,565][44959] Updated weights for policy 1, policy_version 6270 (0.0007) [2023-10-12 20:12:16,849][44958] Updated weights for policy 0, policy_version 6240 (0.0008) [2023-10-12 20:12:20,905][44959] Updated weights for policy 1, policy_version 6280 (0.0009) [2023-10-12 20:12:21,109][44958] Updated weights for policy 0, policy_version 6250 (0.0009) [2023-10-12 20:12:21,278][44959] Updated weights for policy 1, policy_version 6290 (0.0007) [2023-10-12 20:12:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12812288. Throughput: 0: 1638.4, 1: 1640.6. Samples: 3215962. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 20:12:21,443][43579] Avg episode reward: [(0, '220.780'), (1, '221.100')] [2023-10-12 20:12:21,478][44958] Updated weights for policy 0, policy_version 6260 (0.0007) [2023-10-12 20:12:21,649][44959] Updated weights for policy 1, policy_version 6300 (0.0007) [2023-10-12 20:12:21,847][44958] Updated weights for policy 0, policy_version 6270 (0.0008) [2023-10-12 20:12:25,691][44959] Updated weights for policy 1, policy_version 6310 (0.0008) [2023-10-12 20:12:25,996][44958] Updated weights for policy 0, policy_version 6280 (0.0008) [2023-10-12 20:12:26,057][44959] Updated weights for policy 1, policy_version 6320 (0.0007) [2023-10-12 20:12:26,367][44958] Updated weights for policy 0, policy_version 6290 (0.0008) [2023-10-12 20:12:26,427][44959] Updated weights for policy 1, policy_version 6330 (0.0008) [2023-10-12 20:12:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12877824. Throughput: 0: 1637.5, 1: 1643.6. Samples: 3236138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:12:26,443][43579] Avg episode reward: [(0, '226.920'), (1, '229.110')] [2023-10-12 20:12:26,740][44958] Updated weights for policy 0, policy_version 6300 (0.0009) [2023-10-12 20:12:26,883][44518] Saving new best policy, reward=226.920! [2023-10-12 20:12:30,680][44959] Updated weights for policy 1, policy_version 6340 (0.0008) [2023-10-12 20:12:30,928][44958] Updated weights for policy 0, policy_version 6310 (0.0008) [2023-10-12 20:12:31,042][44959] Updated weights for policy 1, policy_version 6350 (0.0007) [2023-10-12 20:12:31,304][44958] Updated weights for policy 0, policy_version 6320 (0.0008) [2023-10-12 20:12:31,405][44959] Updated weights for policy 1, policy_version 6360 (0.0009) [2023-10-12 20:12:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 12943360. Throughput: 0: 1635.0, 1: 1636.4. Samples: 3255320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:12:31,443][43579] Avg episode reward: [(0, '230.580'), (1, '234.710')] [2023-10-12 20:12:31,666][44958] Updated weights for policy 0, policy_version 6330 (0.0007) [2023-10-12 20:12:31,696][44583] Saving new best policy, reward=234.710! [2023-10-12 20:12:31,891][44518] Saving new best policy, reward=230.580! [2023-10-12 20:12:35,600][44959] Updated weights for policy 1, policy_version 6370 (0.0007) [2023-10-12 20:12:35,736][44958] Updated weights for policy 0, policy_version 6340 (0.0009) [2023-10-12 20:12:35,967][44959] Updated weights for policy 1, policy_version 6380 (0.0008) [2023-10-12 20:12:36,100][44958] Updated weights for policy 0, policy_version 6350 (0.0007) [2023-10-12 20:12:36,333][44959] Updated weights for policy 1, policy_version 6390 (0.0009) [2023-10-12 20:12:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13008896. Throughput: 0: 1644.9, 1: 1635.2. Samples: 3265272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:12:36,443][43579] Avg episode reward: [(0, '232.230'), (1, '231.290')] [2023-10-12 20:12:36,475][44958] Updated weights for policy 0, policy_version 6360 (0.0008) [2023-10-12 20:12:36,702][44959] Updated weights for policy 1, policy_version 6400 (0.0008) [2023-10-12 20:12:36,768][44518] Saving new best policy, reward=232.230! [2023-10-12 20:12:40,486][44958] Updated weights for policy 0, policy_version 6370 (0.0007) [2023-10-12 20:12:40,789][44959] Updated weights for policy 1, policy_version 6410 (0.0007) [2023-10-12 20:12:40,853][44958] Updated weights for policy 0, policy_version 6380 (0.0008) [2023-10-12 20:12:41,155][44959] Updated weights for policy 1, policy_version 6420 (0.0007) [2023-10-12 20:12:41,219][44958] Updated weights for policy 0, policy_version 6390 (0.0009) [2023-10-12 20:12:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13074432. Throughput: 0: 1641.1, 1: 1638.1. Samples: 3285506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:12:41,443][43579] Avg episode reward: [(0, '243.920'), (1, '226.110')] [2023-10-12 20:12:41,519][44959] Updated weights for policy 1, policy_version 6430 (0.0007) [2023-10-12 20:12:41,596][44518] Saving new best policy, reward=243.920! [2023-10-12 20:12:41,597][44958] Updated weights for policy 0, policy_version 6400 (0.0010) [2023-10-12 20:12:45,854][44958] Updated weights for policy 0, policy_version 6410 (0.0010) [2023-10-12 20:12:45,882][44959] Updated weights for policy 1, policy_version 6440 (0.0008) [2023-10-12 20:12:46,229][44958] Updated weights for policy 0, policy_version 6420 (0.0009) [2023-10-12 20:12:46,252][44959] Updated weights for policy 1, policy_version 6450 (0.0007) [2023-10-12 20:12:46,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13139968. Throughput: 0: 1634.8, 1: 1639.6. Samples: 3304480. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-12 20:12:46,444][43579] Avg episode reward: [(0, '243.380'), (1, '225.090')] [2023-10-12 20:12:46,603][44958] Updated weights for policy 0, policy_version 6430 (0.0008) [2023-10-12 20:12:46,617][44959] Updated weights for policy 1, policy_version 6460 (0.0010) [2023-10-12 20:12:50,775][44959] Updated weights for policy 1, policy_version 6470 (0.0009) [2023-10-12 20:12:50,824][44958] Updated weights for policy 0, policy_version 6440 (0.0009) [2023-10-12 20:12:51,153][44959] Updated weights for policy 1, policy_version 6480 (0.0008) [2023-10-12 20:12:51,196][44958] Updated weights for policy 0, policy_version 6450 (0.0009) [2023-10-12 20:12:51,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 13205504. Throughput: 0: 1640.8, 1: 1636.3. Samples: 3314322. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-12 20:12:51,444][43579] Avg episode reward: [(0, '238.150'), (1, '218.170')] [2023-10-12 20:12:51,522][44959] Updated weights for policy 1, policy_version 6490 (0.0009) [2023-10-12 20:12:51,571][44958] Updated weights for policy 0, policy_version 6460 (0.0009) [2023-10-12 20:12:55,516][44959] Updated weights for policy 1, policy_version 6500 (0.0008) [2023-10-12 20:12:55,856][44958] Updated weights for policy 0, policy_version 6470 (0.0008) [2023-10-12 20:12:55,896][44959] Updated weights for policy 1, policy_version 6510 (0.0009) [2023-10-12 20:12:56,244][44958] Updated weights for policy 0, policy_version 6480 (0.0007) [2023-10-12 20:12:56,258][44959] Updated weights for policy 1, policy_version 6520 (0.0009) [2023-10-12 20:12:56,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 13271040. Throughput: 0: 1635.5, 1: 1647.1. Samples: 3334782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:12:56,444][43579] Avg episode reward: [(0, '235.930'), (1, '214.100')] [2023-10-12 20:12:56,608][44958] Updated weights for policy 0, policy_version 6490 (0.0007) [2023-10-12 20:13:00,304][44959] Updated weights for policy 1, policy_version 6530 (0.0009) [2023-10-12 20:13:00,670][44959] Updated weights for policy 1, policy_version 6540 (0.0009) [2023-10-12 20:13:00,683][44958] Updated weights for policy 0, policy_version 6500 (0.0008) [2023-10-12 20:13:01,042][44959] Updated weights for policy 1, policy_version 6550 (0.0009) [2023-10-12 20:13:01,066][44958] Updated weights for policy 0, policy_version 6510 (0.0008) [2023-10-12 20:13:01,415][44959] Updated weights for policy 1, policy_version 6560 (0.0008) [2023-10-12 20:13:01,442][43579] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 13369344. Throughput: 0: 1638.4, 1: 1639.2. Samples: 3353582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:13:01,443][43579] Avg episode reward: [(0, '236.130'), (1, '211.570')] [2023-10-12 20:13:01,447][44958] Updated weights for policy 0, policy_version 6520 (0.0008) [2023-10-12 20:13:05,463][44958] Updated weights for policy 0, policy_version 6530 (0.0008) [2023-10-12 20:13:05,682][44959] Updated weights for policy 1, policy_version 6570 (0.0010) [2023-10-12 20:13:05,833][44958] Updated weights for policy 0, policy_version 6540 (0.0008) [2023-10-12 20:13:06,058][44959] Updated weights for policy 1, policy_version 6580 (0.0007) [2023-10-12 20:13:06,203][44958] Updated weights for policy 0, policy_version 6550 (0.0009) [2023-10-12 20:13:06,421][44959] Updated weights for policy 1, policy_version 6590 (0.0007) [2023-10-12 20:13:06,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 13402112. Throughput: 0: 1643.3, 1: 1643.8. Samples: 3363882. Policy #0 lag: (min: 0.0, avg: 14.6, max: 32.0) [2023-10-12 20:13:06,443][43579] Avg episode reward: [(0, '233.480'), (1, '216.580')] [2023-10-12 20:13:06,577][44958] Updated weights for policy 0, policy_version 6560 (0.0008) [2023-10-12 20:13:10,685][44959] Updated weights for policy 1, policy_version 6600 (0.0007) [2023-10-12 20:13:10,809][44958] Updated weights for policy 0, policy_version 6570 (0.0008) [2023-10-12 20:13:11,044][44959] Updated weights for policy 1, policy_version 6610 (0.0007) [2023-10-12 20:13:11,180][44958] Updated weights for policy 0, policy_version 6580 (0.0007) [2023-10-12 20:13:11,423][44959] Updated weights for policy 1, policy_version 6620 (0.0007) [2023-10-12 20:13:11,443][43579] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13467648. Throughput: 0: 1647.0, 1: 1637.0. Samples: 3383916. Policy #0 lag: (min: 0.0, avg: 14.6, max: 32.0) [2023-10-12 20:13:11,443][43579] Avg episode reward: [(0, '235.040'), (1, '220.280')] [2023-10-12 20:13:11,545][44958] Updated weights for policy 0, policy_version 6590 (0.0007) [2023-10-12 20:13:15,655][44959] Updated weights for policy 1, policy_version 6630 (0.0007) [2023-10-12 20:13:15,724][44958] Updated weights for policy 0, policy_version 6600 (0.0009) [2023-10-12 20:13:16,028][44959] Updated weights for policy 1, policy_version 6640 (0.0008) [2023-10-12 20:13:16,091][44958] Updated weights for policy 0, policy_version 6610 (0.0008) [2023-10-12 20:13:16,390][44959] Updated weights for policy 1, policy_version 6650 (0.0008) [2023-10-12 20:13:16,443][43579] Fps is (10 sec: 13106.5, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 13533184. Throughput: 0: 1640.4, 1: 1635.1. Samples: 3402720. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-12 20:13:16,444][43579] Avg episode reward: [(0, '235.490'), (1, '224.060')] [2023-10-12 20:13:16,466][44958] Updated weights for policy 0, policy_version 6620 (0.0008) [2023-10-12 20:13:20,588][44959] Updated weights for policy 1, policy_version 6660 (0.0008) [2023-10-12 20:13:20,771][44958] Updated weights for policy 0, policy_version 6630 (0.0008) [2023-10-12 20:13:20,964][44959] Updated weights for policy 1, policy_version 6670 (0.0009) [2023-10-12 20:13:21,147][44958] Updated weights for policy 0, policy_version 6640 (0.0008) [2023-10-12 20:13:21,328][44959] Updated weights for policy 1, policy_version 6680 (0.0007) [2023-10-12 20:13:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13598720. Throughput: 0: 1638.8, 1: 1637.8. Samples: 3412722. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-12 20:13:21,443][43579] Avg episode reward: [(0, '241.570'), (1, '225.380')] [2023-10-12 20:13:21,520][44958] Updated weights for policy 0, policy_version 6650 (0.0008) [2023-10-12 20:13:25,683][44959] Updated weights for policy 1, policy_version 6690 (0.0008) [2023-10-12 20:13:25,778][44958] Updated weights for policy 0, policy_version 6660 (0.0009) [2023-10-12 20:13:26,046][44959] Updated weights for policy 1, policy_version 6700 (0.0007) [2023-10-12 20:13:26,145][44958] Updated weights for policy 0, policy_version 6670 (0.0009) [2023-10-12 20:13:26,411][44959] Updated weights for policy 1, policy_version 6710 (0.0009) [2023-10-12 20:13:26,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13664256. Throughput: 0: 1640.4, 1: 1635.9. Samples: 3432940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:13:26,443][43579] Avg episode reward: [(0, '241.160'), (1, '229.910')] [2023-10-12 20:13:26,517][44958] Updated weights for policy 0, policy_version 6680 (0.0007) [2023-10-12 20:13:26,784][44959] Updated weights for policy 1, policy_version 6720 (0.0008) [2023-10-12 20:13:30,580][44958] Updated weights for policy 0, policy_version 6690 (0.0008) [2023-10-12 20:13:30,759][44959] Updated weights for policy 1, policy_version 6730 (0.0007) [2023-10-12 20:13:30,959][44958] Updated weights for policy 0, policy_version 6700 (0.0008) [2023-10-12 20:13:31,138][44959] Updated weights for policy 1, policy_version 6740 (0.0007) [2023-10-12 20:13:31,327][44958] Updated weights for policy 0, policy_version 6710 (0.0007) [2023-10-12 20:13:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13729792. Throughput: 0: 1638.1, 1: 1638.1. Samples: 3451908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:13:31,443][43579] Avg episode reward: [(0, '242.790'), (1, '225.200')] [2023-10-12 20:13:31,512][44959] Updated weights for policy 1, policy_version 6750 (0.0008) [2023-10-12 20:13:31,695][44958] Updated weights for policy 0, policy_version 6720 (0.0007) [2023-10-12 20:13:35,633][44959] Updated weights for policy 1, policy_version 6760 (0.0007) [2023-10-12 20:13:36,008][44959] Updated weights for policy 1, policy_version 6770 (0.0007) [2023-10-12 20:13:36,015][44958] Updated weights for policy 0, policy_version 6730 (0.0008) [2023-10-12 20:13:36,376][44959] Updated weights for policy 1, policy_version 6780 (0.0009) [2023-10-12 20:13:36,390][44958] Updated weights for policy 0, policy_version 6740 (0.0008) [2023-10-12 20:13:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13795328. Throughput: 0: 1637.3, 1: 1644.1. Samples: 3461988. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 20:13:36,444][43579] Avg episode reward: [(0, '242.250'), (1, '222.790')] [2023-10-12 20:13:36,752][44958] Updated weights for policy 0, policy_version 6750 (0.0008) [2023-10-12 20:13:40,518][44959] Updated weights for policy 1, policy_version 6790 (0.0007) [2023-10-12 20:13:40,890][44959] Updated weights for policy 1, policy_version 6800 (0.0009) [2023-10-12 20:13:40,996][44958] Updated weights for policy 0, policy_version 6760 (0.0007) [2023-10-12 20:13:41,254][44959] Updated weights for policy 1, policy_version 6810 (0.0009) [2023-10-12 20:13:41,380][44958] Updated weights for policy 0, policy_version 6770 (0.0008) [2023-10-12 20:13:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 13860864. Throughput: 0: 1637.1, 1: 1639.0. Samples: 3482206. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 20:13:41,443][43579] Avg episode reward: [(0, '241.630'), (1, '218.340')] [2023-10-12 20:13:41,742][44958] Updated weights for policy 0, policy_version 6780 (0.0009) [2023-10-12 20:13:45,190][44959] Updated weights for policy 1, policy_version 6820 (0.0009) [2023-10-12 20:13:45,550][44959] Updated weights for policy 1, policy_version 6830 (0.0009) [2023-10-12 20:13:45,927][44959] Updated weights for policy 1, policy_version 6840 (0.0009) [2023-10-12 20:13:45,989][44958] Updated weights for policy 0, policy_version 6790 (0.0008) [2023-10-12 20:13:46,357][44958] Updated weights for policy 0, policy_version 6800 (0.0011) [2023-10-12 20:13:46,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 13959168. Throughput: 0: 1638.3, 1: 1641.6. Samples: 3501178. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) [2023-10-12 20:13:46,444][43579] Avg episode reward: [(0, '238.660'), (1, '218.790')] [2023-10-12 20:13:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000006848_7012352.pth... [2023-10-12 20:13:46,485][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000005312_5439488.pth [2023-10-12 20:13:46,723][44958] Updated weights for policy 0, policy_version 6810 (0.0010) [2023-10-12 20:13:46,944][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000006816_6979584.pth... [2023-10-12 20:13:46,984][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000005280_5406720.pth [2023-10-12 20:13:50,118][44959] Updated weights for policy 1, policy_version 6850 (0.0008) [2023-10-12 20:13:50,489][44959] Updated weights for policy 1, policy_version 6860 (0.0007) [2023-10-12 20:13:50,856][44959] Updated weights for policy 1, policy_version 6870 (0.0007) [2023-10-12 20:13:50,968][44958] Updated weights for policy 0, policy_version 6820 (0.0008) [2023-10-12 20:13:51,228][44959] Updated weights for policy 1, policy_version 6880 (0.0007) [2023-10-12 20:13:51,334][44958] Updated weights for policy 0, policy_version 6830 (0.0007) [2023-10-12 20:13:51,442][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 14024704. Throughput: 0: 1631.3, 1: 1643.6. Samples: 3511256. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) [2023-10-12 20:13:51,445][43579] Avg episode reward: [(0, '239.030'), (1, '214.590')] [2023-10-12 20:13:51,705][44958] Updated weights for policy 0, policy_version 6840 (0.0007) [2023-10-12 20:13:55,420][44959] Updated weights for policy 1, policy_version 6890 (0.0008) [2023-10-12 20:13:55,775][44959] Updated weights for policy 1, policy_version 6900 (0.0009) [2023-10-12 20:13:55,944][44958] Updated weights for policy 0, policy_version 6850 (0.0008) [2023-10-12 20:13:56,148][44959] Updated weights for policy 1, policy_version 6910 (0.0009) [2023-10-12 20:13:56,323][44958] Updated weights for policy 0, policy_version 6860 (0.0009) [2023-10-12 20:13:56,442][43579] Fps is (10 sec: 13107.8, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 14090240. Throughput: 0: 1627.4, 1: 1651.1. Samples: 3531450. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-10-12 20:13:56,443][43579] Avg episode reward: [(0, '238.950'), (1, '215.130')] [2023-10-12 20:13:56,690][44958] Updated weights for policy 0, policy_version 6870 (0.0007) [2023-10-12 20:13:57,056][44958] Updated weights for policy 0, policy_version 6880 (0.0008) [2023-10-12 20:14:00,353][44959] Updated weights for policy 1, policy_version 6920 (0.0010) [2023-10-12 20:14:00,728][44959] Updated weights for policy 1, policy_version 6930 (0.0011) [2023-10-12 20:14:01,070][44958] Updated weights for policy 0, policy_version 6890 (0.0009) [2023-10-12 20:14:01,102][44959] Updated weights for policy 1, policy_version 6940 (0.0008) [2023-10-12 20:14:01,440][44958] Updated weights for policy 0, policy_version 6900 (0.0009) [2023-10-12 20:14:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 14155776. Throughput: 0: 1636.7, 1: 1645.7. Samples: 3550428. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-10-12 20:14:01,444][43579] Avg episode reward: [(0, '242.010'), (1, '223.060')] [2023-10-12 20:14:01,819][44958] Updated weights for policy 0, policy_version 6910 (0.0009) [2023-10-12 20:14:05,171][44959] Updated weights for policy 1, policy_version 6950 (0.0008) [2023-10-12 20:14:05,524][44959] Updated weights for policy 1, policy_version 6960 (0.0008) [2023-10-12 20:14:05,889][44959] Updated weights for policy 1, policy_version 6970 (0.0008) [2023-10-12 20:14:05,961][44958] Updated weights for policy 0, policy_version 6920 (0.0009) [2023-10-12 20:14:06,336][44958] Updated weights for policy 0, policy_version 6930 (0.0010) [2023-10-12 20:14:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 14221312. Throughput: 0: 1632.0, 1: 1657.4. Samples: 3560744. Policy #0 lag: (min: 39.0, avg: 54.6, max: 56.0) [2023-10-12 20:14:06,444][43579] Avg episode reward: [(0, '250.200'), (1, '215.750')] [2023-10-12 20:14:06,714][44958] Updated weights for policy 0, policy_version 6940 (0.0009) [2023-10-12 20:14:06,860][44518] Saving new best policy, reward=250.200! [2023-10-12 20:14:09,870][44959] Updated weights for policy 1, policy_version 6980 (0.0009) [2023-10-12 20:14:10,242][44959] Updated weights for policy 1, policy_version 6990 (0.0007) [2023-10-12 20:14:10,616][44959] Updated weights for policy 1, policy_version 7000 (0.0009) [2023-10-12 20:14:10,721][44958] Updated weights for policy 0, policy_version 6950 (0.0009) [2023-10-12 20:14:11,089][44958] Updated weights for policy 0, policy_version 6960 (0.0008) [2023-10-12 20:14:11,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 14286848. Throughput: 0: 1630.3, 1: 1659.7. Samples: 3580992. Policy #0 lag: (min: 39.0, avg: 54.6, max: 56.0) [2023-10-12 20:14:11,443][43579] Avg episode reward: [(0, '247.480'), (1, '223.470')] [2023-10-12 20:14:11,460][44958] Updated weights for policy 0, policy_version 6970 (0.0008) [2023-10-12 20:14:14,873][44959] Updated weights for policy 1, policy_version 7010 (0.0010) [2023-10-12 20:14:15,243][44959] Updated weights for policy 1, policy_version 7020 (0.0009) [2023-10-12 20:14:15,612][44959] Updated weights for policy 1, policy_version 7030 (0.0008) [2023-10-12 20:14:15,659][44958] Updated weights for policy 0, policy_version 6980 (0.0010) [2023-10-12 20:14:15,984][44959] Updated weights for policy 1, policy_version 7040 (0.0007) [2023-10-12 20:14:16,028][44958] Updated weights for policy 0, policy_version 6990 (0.0008) [2023-10-12 20:14:16,392][44958] Updated weights for policy 0, policy_version 7000 (0.0011) [2023-10-12 20:14:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 14352384. Throughput: 0: 1636.4, 1: 1651.4. Samples: 3599860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:14:16,444][43579] Avg episode reward: [(0, '250.480'), (1, '224.710')] [2023-10-12 20:14:16,685][44518] Saving new best policy, reward=250.480! [2023-10-12 20:14:20,099][44959] Updated weights for policy 1, policy_version 7050 (0.0008) [2023-10-12 20:14:20,464][44959] Updated weights for policy 1, policy_version 7060 (0.0008) [2023-10-12 20:14:20,654][44958] Updated weights for policy 0, policy_version 7010 (0.0011) [2023-10-12 20:14:20,833][44959] Updated weights for policy 1, policy_version 7070 (0.0008) [2023-10-12 20:14:21,022][44958] Updated weights for policy 0, policy_version 7020 (0.0009) [2023-10-12 20:14:21,392][44958] Updated weights for policy 0, policy_version 7030 (0.0010) [2023-10-12 20:14:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 14417920. Throughput: 0: 1638.4, 1: 1664.1. Samples: 3610596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:14:21,443][43579] Avg episode reward: [(0, '249.060'), (1, '229.280')] [2023-10-12 20:14:21,769][44958] Updated weights for policy 0, policy_version 7040 (0.0009) [2023-10-12 20:14:25,065][44959] Updated weights for policy 1, policy_version 7080 (0.0008) [2023-10-12 20:14:25,438][44959] Updated weights for policy 1, policy_version 7090 (0.0007) [2023-10-12 20:14:25,803][44959] Updated weights for policy 1, policy_version 7100 (0.0007) [2023-10-12 20:14:25,977][44958] Updated weights for policy 0, policy_version 7050 (0.0007) [2023-10-12 20:14:26,360][44958] Updated weights for policy 0, policy_version 7060 (0.0007) [2023-10-12 20:14:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 14483456. Throughput: 0: 1640.7, 1: 1657.2. Samples: 3630610. Policy #0 lag: (min: 31.0, avg: 32.2, max: 56.0) [2023-10-12 20:14:26,444][43579] Avg episode reward: [(0, '248.640'), (1, '220.780')] [2023-10-12 20:14:26,734][44958] Updated weights for policy 0, policy_version 7070 (0.0009) [2023-10-12 20:14:29,799][44959] Updated weights for policy 1, policy_version 7110 (0.0007) [2023-10-12 20:14:30,166][44959] Updated weights for policy 1, policy_version 7120 (0.0009) [2023-10-12 20:14:30,531][44959] Updated weights for policy 1, policy_version 7130 (0.0009) [2023-10-12 20:14:30,858][44958] Updated weights for policy 0, policy_version 7080 (0.0009) [2023-10-12 20:14:31,233][44958] Updated weights for policy 0, policy_version 7090 (0.0010) [2023-10-12 20:14:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 14548992. Throughput: 0: 1638.3, 1: 1655.7. Samples: 3649406. Policy #0 lag: (min: 31.0, avg: 32.2, max: 56.0) [2023-10-12 20:14:31,444][43579] Avg episode reward: [(0, '246.010'), (1, '220.970')] [2023-10-12 20:14:31,610][44958] Updated weights for policy 0, policy_version 7100 (0.0008) [2023-10-12 20:14:34,734][44959] Updated weights for policy 1, policy_version 7140 (0.0008) [2023-10-12 20:14:35,107][44959] Updated weights for policy 1, policy_version 7150 (0.0007) [2023-10-12 20:14:35,484][44959] Updated weights for policy 1, policy_version 7160 (0.0008) [2023-10-12 20:14:35,793][44958] Updated weights for policy 0, policy_version 7110 (0.0009) [2023-10-12 20:14:36,156][44958] Updated weights for policy 0, policy_version 7120 (0.0008) [2023-10-12 20:14:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 14614528. Throughput: 0: 1640.5, 1: 1665.0. Samples: 3660006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:14:36,443][43579] Avg episode reward: [(0, '245.860'), (1, '227.160')] [2023-10-12 20:14:36,536][44958] Updated weights for policy 0, policy_version 7130 (0.0009) [2023-10-12 20:14:39,714][44959] Updated weights for policy 1, policy_version 7170 (0.0008) [2023-10-12 20:14:40,129][44959] Updated weights for policy 1, policy_version 7180 (0.0008) [2023-10-12 20:14:40,508][44959] Updated weights for policy 1, policy_version 7190 (0.0009) [2023-10-12 20:14:40,608][44958] Updated weights for policy 0, policy_version 7140 (0.0008) [2023-10-12 20:14:40,881][44959] Updated weights for policy 1, policy_version 7200 (0.0007) [2023-10-12 20:14:40,991][44958] Updated weights for policy 0, policy_version 7150 (0.0009) [2023-10-12 20:14:41,367][44958] Updated weights for policy 0, policy_version 7160 (0.0010) [2023-10-12 20:14:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 14680064. Throughput: 0: 1642.5, 1: 1651.6. Samples: 3679684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:14:41,443][43579] Avg episode reward: [(0, '249.300'), (1, '223.230')] [2023-10-12 20:14:44,991][44959] Updated weights for policy 1, policy_version 7210 (0.0008) [2023-10-12 20:14:45,350][44959] Updated weights for policy 1, policy_version 7220 (0.0008) [2023-10-12 20:14:45,521][44958] Updated weights for policy 0, policy_version 7170 (0.0009) [2023-10-12 20:14:45,722][44959] Updated weights for policy 1, policy_version 7230 (0.0009) [2023-10-12 20:14:45,895][44958] Updated weights for policy 0, policy_version 7180 (0.0009) [2023-10-12 20:14:46,273][44958] Updated weights for policy 0, policy_version 7190 (0.0011) [2023-10-12 20:14:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 14745600. Throughput: 0: 1634.2, 1: 1653.5. Samples: 3698374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:14:46,443][43579] Avg episode reward: [(0, '249.210'), (1, '225.770')] [2023-10-12 20:14:46,650][44958] Updated weights for policy 0, policy_version 7200 (0.0009) [2023-10-12 20:14:49,915][44959] Updated weights for policy 1, policy_version 7240 (0.0010) [2023-10-12 20:14:50,298][44959] Updated weights for policy 1, policy_version 7250 (0.0010) [2023-10-12 20:14:50,665][44959] Updated weights for policy 1, policy_version 7260 (0.0009) [2023-10-12 20:14:50,950][44958] Updated weights for policy 0, policy_version 7210 (0.0009) [2023-10-12 20:14:51,321][44958] Updated weights for policy 0, policy_version 7220 (0.0009) [2023-10-12 20:14:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 14811136. Throughput: 0: 1642.4, 1: 1654.3. Samples: 3709096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:14:51,443][43579] Avg episode reward: [(0, '255.090'), (1, '231.540')] [2023-10-12 20:14:51,692][44958] Updated weights for policy 0, policy_version 7230 (0.0008) [2023-10-12 20:14:51,764][44518] Saving new best policy, reward=255.090! [2023-10-12 20:14:54,832][44959] Updated weights for policy 1, policy_version 7270 (0.0008) [2023-10-12 20:14:55,200][44959] Updated weights for policy 1, policy_version 7280 (0.0009) [2023-10-12 20:14:55,559][44959] Updated weights for policy 1, policy_version 7290 (0.0008) [2023-10-12 20:14:56,027][44958] Updated weights for policy 0, policy_version 7240 (0.0008) [2023-10-12 20:14:56,393][44958] Updated weights for policy 0, policy_version 7250 (0.0007) [2023-10-12 20:14:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 14876672. Throughput: 0: 1639.4, 1: 1642.7. Samples: 3728688. Policy #0 lag: (min: 29.0, avg: 36.6, max: 61.0) [2023-10-12 20:14:56,443][43579] Avg episode reward: [(0, '256.810'), (1, '232.050')] [2023-10-12 20:14:56,771][44958] Updated weights for policy 0, policy_version 7260 (0.0009) [2023-10-12 20:14:56,913][44518] Saving new best policy, reward=256.810! [2023-10-12 20:14:59,745][44959] Updated weights for policy 1, policy_version 7300 (0.0009) [2023-10-12 20:15:00,122][44959] Updated weights for policy 1, policy_version 7310 (0.0008) [2023-10-12 20:15:00,488][44959] Updated weights for policy 1, policy_version 7320 (0.0007) [2023-10-12 20:15:00,898][44958] Updated weights for policy 0, policy_version 7270 (0.0009) [2023-10-12 20:15:01,268][44958] Updated weights for policy 0, policy_version 7280 (0.0011) [2023-10-12 20:15:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 14942208. Throughput: 0: 1638.0, 1: 1644.9. Samples: 3747592. Policy #0 lag: (min: 29.0, avg: 36.6, max: 61.0) [2023-10-12 20:15:01,443][43579] Avg episode reward: [(0, '257.570'), (1, '226.970')] [2023-10-12 20:15:01,648][44958] Updated weights for policy 0, policy_version 7290 (0.0009) [2023-10-12 20:15:01,865][44518] Saving new best policy, reward=257.570! [2023-10-12 20:15:04,558][44959] Updated weights for policy 1, policy_version 7330 (0.0007) [2023-10-12 20:15:04,921][44959] Updated weights for policy 1, policy_version 7340 (0.0010) [2023-10-12 20:15:05,295][44959] Updated weights for policy 1, policy_version 7350 (0.0007) [2023-10-12 20:15:05,670][44959] Updated weights for policy 1, policy_version 7360 (0.0007) [2023-10-12 20:15:05,865][44958] Updated weights for policy 0, policy_version 7300 (0.0007) [2023-10-12 20:15:06,245][44958] Updated weights for policy 0, policy_version 7310 (0.0009) [2023-10-12 20:15:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15007744. Throughput: 0: 1633.4, 1: 1649.4. Samples: 3758320. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-12 20:15:06,444][43579] Avg episode reward: [(0, '253.900'), (1, '225.860')] [2023-10-12 20:15:06,619][44958] Updated weights for policy 0, policy_version 7320 (0.0008) [2023-10-12 20:15:09,696][44959] Updated weights for policy 1, policy_version 7370 (0.0008) [2023-10-12 20:15:10,062][44959] Updated weights for policy 1, policy_version 7380 (0.0009) [2023-10-12 20:15:10,427][44959] Updated weights for policy 1, policy_version 7390 (0.0008) [2023-10-12 20:15:10,823][44958] Updated weights for policy 0, policy_version 7330 (0.0010) [2023-10-12 20:15:11,226][44958] Updated weights for policy 0, policy_version 7340 (0.0008) [2023-10-12 20:15:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15073280. Throughput: 0: 1633.0, 1: 1642.3. Samples: 3777998. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-12 20:15:11,443][43579] Avg episode reward: [(0, '250.490'), (1, '230.140')] [2023-10-12 20:15:11,608][44958] Updated weights for policy 0, policy_version 7350 (0.0007) [2023-10-12 20:15:11,987][44958] Updated weights for policy 0, policy_version 7360 (0.0010) [2023-10-12 20:15:14,692][44959] Updated weights for policy 1, policy_version 7400 (0.0008) [2023-10-12 20:15:15,062][44959] Updated weights for policy 1, policy_version 7410 (0.0008) [2023-10-12 20:15:15,448][44959] Updated weights for policy 1, policy_version 7420 (0.0010) [2023-10-12 20:15:15,943][44958] Updated weights for policy 0, policy_version 7370 (0.0010) [2023-10-12 20:15:16,315][44958] Updated weights for policy 0, policy_version 7380 (0.0011) [2023-10-12 20:15:16,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15138816. Throughput: 0: 1633.3, 1: 1642.1. Samples: 3796794. Policy #0 lag: (min: 13.0, avg: 16.0, max: 45.0) [2023-10-12 20:15:16,443][43579] Avg episode reward: [(0, '247.600'), (1, '228.550')] [2023-10-12 20:15:16,687][44958] Updated weights for policy 0, policy_version 7390 (0.0010) [2023-10-12 20:15:19,616][44959] Updated weights for policy 1, policy_version 7430 (0.0009) [2023-10-12 20:15:19,990][44959] Updated weights for policy 1, policy_version 7440 (0.0010) [2023-10-12 20:15:20,355][44959] Updated weights for policy 1, policy_version 7450 (0.0010) [2023-10-12 20:15:20,888][44958] Updated weights for policy 0, policy_version 7400 (0.0007) [2023-10-12 20:15:21,274][44958] Updated weights for policy 0, policy_version 7410 (0.0007) [2023-10-12 20:15:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15204352. Throughput: 0: 1641.2, 1: 1640.7. Samples: 3807694. Policy #0 lag: (min: 13.0, avg: 16.0, max: 45.0) [2023-10-12 20:15:21,444][43579] Avg episode reward: [(0, '243.570'), (1, '232.460')] [2023-10-12 20:15:21,643][44958] Updated weights for policy 0, policy_version 7420 (0.0011) [2023-10-12 20:15:24,563][44959] Updated weights for policy 1, policy_version 7460 (0.0008) [2023-10-12 20:15:24,957][44959] Updated weights for policy 1, policy_version 7470 (0.0010) [2023-10-12 20:15:25,331][44959] Updated weights for policy 1, policy_version 7480 (0.0010) [2023-10-12 20:15:25,774][44958] Updated weights for policy 0, policy_version 7430 (0.0008) [2023-10-12 20:15:26,150][44958] Updated weights for policy 0, policy_version 7440 (0.0007) [2023-10-12 20:15:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15269888. Throughput: 0: 1639.4, 1: 1636.3. Samples: 3827094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:15:26,444][43579] Avg episode reward: [(0, '239.910'), (1, '229.870')] [2023-10-12 20:15:26,529][44958] Updated weights for policy 0, policy_version 7450 (0.0008) [2023-10-12 20:15:29,567][44959] Updated weights for policy 1, policy_version 7490 (0.0010) [2023-10-12 20:15:29,930][44959] Updated weights for policy 1, policy_version 7500 (0.0011) [2023-10-12 20:15:30,306][44959] Updated weights for policy 1, policy_version 7510 (0.0008) [2023-10-12 20:15:30,678][44959] Updated weights for policy 1, policy_version 7520 (0.0009) [2023-10-12 20:15:30,747][44958] Updated weights for policy 0, policy_version 7460 (0.0010) [2023-10-12 20:15:31,122][44958] Updated weights for policy 0, policy_version 7470 (0.0008) [2023-10-12 20:15:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 15335424. Throughput: 0: 1643.3, 1: 1642.1. Samples: 3846216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:15:31,443][43579] Avg episode reward: [(0, '240.530'), (1, '240.550')] [2023-10-12 20:15:31,451][44583] Saving new best policy, reward=240.550! [2023-10-12 20:15:31,490][44958] Updated weights for policy 0, policy_version 7480 (0.0009) [2023-10-12 20:15:34,818][44959] Updated weights for policy 1, policy_version 7530 (0.0010) [2023-10-12 20:15:35,187][44959] Updated weights for policy 1, policy_version 7540 (0.0009) [2023-10-12 20:15:35,551][44959] Updated weights for policy 1, policy_version 7550 (0.0009) [2023-10-12 20:15:35,599][44958] Updated weights for policy 0, policy_version 7490 (0.0009) [2023-10-12 20:15:35,977][44958] Updated weights for policy 0, policy_version 7500 (0.0011) [2023-10-12 20:15:36,337][44958] Updated weights for policy 0, policy_version 7510 (0.0011) [2023-10-12 20:15:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15400960. Throughput: 0: 1640.8, 1: 1643.6. Samples: 3856894. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-12 20:15:36,443][43579] Avg episode reward: [(0, '237.520'), (1, '241.980')] [2023-10-12 20:15:36,444][44583] Saving new best policy, reward=241.980! [2023-10-12 20:15:36,712][44958] Updated weights for policy 0, policy_version 7520 (0.0008) [2023-10-12 20:15:39,622][44959] Updated weights for policy 1, policy_version 7560 (0.0008) [2023-10-12 20:15:39,987][44959] Updated weights for policy 1, policy_version 7570 (0.0008) [2023-10-12 20:15:40,370][44959] Updated weights for policy 1, policy_version 7580 (0.0009) [2023-10-12 20:15:41,018][44958] Updated weights for policy 0, policy_version 7530 (0.0008) [2023-10-12 20:15:41,399][44958] Updated weights for policy 0, policy_version 7540 (0.0008) [2023-10-12 20:15:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15466496. Throughput: 0: 1644.4, 1: 1643.2. Samples: 3876630. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-12 20:15:41,443][43579] Avg episode reward: [(0, '232.430'), (1, '231.290')] [2023-10-12 20:15:41,764][44958] Updated weights for policy 0, policy_version 7550 (0.0007) [2023-10-12 20:15:44,631][44959] Updated weights for policy 1, policy_version 7590 (0.0009) [2023-10-12 20:15:44,996][44959] Updated weights for policy 1, policy_version 7600 (0.0007) [2023-10-12 20:15:45,361][44959] Updated weights for policy 1, policy_version 7610 (0.0008) [2023-10-12 20:15:45,856][44958] Updated weights for policy 0, policy_version 7560 (0.0007) [2023-10-12 20:15:46,237][44958] Updated weights for policy 0, policy_version 7570 (0.0007) [2023-10-12 20:15:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15532032. Throughput: 0: 1643.2, 1: 1649.9. Samples: 3895780. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-12 20:15:46,443][43579] Avg episode reward: [(0, '231.350'), (1, '241.770')] [2023-10-12 20:15:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000007616_7798784.pth... [2023-10-12 20:15:46,483][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000006080_6225920.pth [2023-10-12 20:15:46,487][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000007616_7798784.pth [2023-10-12 20:15:46,607][44958] Updated weights for policy 0, policy_version 7580 (0.0007) [2023-10-12 20:15:46,755][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000007584_7766016.pth... [2023-10-12 20:15:46,784][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000006048_6193152.pth [2023-10-12 20:15:46,788][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000007584_7766016.pth [2023-10-12 20:15:49,636][44959] Updated weights for policy 1, policy_version 7620 (0.0008) [2023-10-12 20:15:50,007][44959] Updated weights for policy 1, policy_version 7630 (0.0007) [2023-10-12 20:15:50,374][44959] Updated weights for policy 1, policy_version 7640 (0.0007) [2023-10-12 20:15:50,921][44958] Updated weights for policy 0, policy_version 7590 (0.0009) [2023-10-12 20:15:51,295][44958] Updated weights for policy 0, policy_version 7600 (0.0009) [2023-10-12 20:15:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 15597568. Throughput: 0: 1644.2, 1: 1642.8. Samples: 3906236. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-12 20:15:51,444][43579] Avg episode reward: [(0, '235.390'), (1, '245.410')] [2023-10-12 20:15:51,445][44583] Saving new best policy, reward=245.410! [2023-10-12 20:15:51,675][44958] Updated weights for policy 0, policy_version 7610 (0.0009) [2023-10-12 20:15:54,571][44959] Updated weights for policy 1, policy_version 7650 (0.0007) [2023-10-12 20:15:54,940][44959] Updated weights for policy 1, policy_version 7660 (0.0008) [2023-10-12 20:15:55,314][44959] Updated weights for policy 1, policy_version 7670 (0.0008) [2023-10-12 20:15:55,687][44959] Updated weights for policy 1, policy_version 7680 (0.0009) [2023-10-12 20:15:55,694][44958] Updated weights for policy 0, policy_version 7620 (0.0009) [2023-10-12 20:15:56,098][44958] Updated weights for policy 0, policy_version 7630 (0.0008) [2023-10-12 20:15:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15663104. Throughput: 0: 1649.6, 1: 1641.3. Samples: 3926090. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 20:15:56,444][43579] Avg episode reward: [(0, '239.380'), (1, '249.510')] [2023-10-12 20:15:56,445][44583] Saving new best policy, reward=249.510! [2023-10-12 20:15:56,471][44958] Updated weights for policy 0, policy_version 7640 (0.0009) [2023-10-12 20:15:59,759][44959] Updated weights for policy 1, policy_version 7690 (0.0007) [2023-10-12 20:16:00,123][44959] Updated weights for policy 1, policy_version 7700 (0.0007) [2023-10-12 20:16:00,496][44959] Updated weights for policy 1, policy_version 7710 (0.0007) [2023-10-12 20:16:00,627][44958] Updated weights for policy 0, policy_version 7650 (0.0007) [2023-10-12 20:16:00,999][44958] Updated weights for policy 0, policy_version 7660 (0.0007) [2023-10-12 20:16:01,374][44958] Updated weights for policy 0, policy_version 7670 (0.0007) [2023-10-12 20:16:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15728640. Throughput: 0: 1647.8, 1: 1643.7. Samples: 3944912. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 20:16:01,444][43579] Avg episode reward: [(0, '248.980'), (1, '251.670')] [2023-10-12 20:16:01,456][44583] Saving new best policy, reward=251.670! [2023-10-12 20:16:01,738][44958] Updated weights for policy 0, policy_version 7680 (0.0007) [2023-10-12 20:16:04,448][44959] Updated weights for policy 1, policy_version 7720 (0.0008) [2023-10-12 20:16:04,809][44959] Updated weights for policy 1, policy_version 7730 (0.0010) [2023-10-12 20:16:05,175][44959] Updated weights for policy 1, policy_version 7740 (0.0007) [2023-10-12 20:16:05,835][44958] Updated weights for policy 0, policy_version 7690 (0.0008) [2023-10-12 20:16:06,210][44958] Updated weights for policy 0, policy_version 7700 (0.0008) [2023-10-12 20:16:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15794176. Throughput: 0: 1646.1, 1: 1648.0. Samples: 3955930. Policy #0 lag: (min: 25.0, avg: 29.9, max: 56.0) [2023-10-12 20:16:06,443][43579] Avg episode reward: [(0, '250.900'), (1, '250.730')] [2023-10-12 20:16:06,591][44958] Updated weights for policy 0, policy_version 7710 (0.0007) [2023-10-12 20:16:09,454][44959] Updated weights for policy 1, policy_version 7750 (0.0008) [2023-10-12 20:16:09,851][44959] Updated weights for policy 1, policy_version 7760 (0.0009) [2023-10-12 20:16:10,224][44959] Updated weights for policy 1, policy_version 7770 (0.0008) [2023-10-12 20:16:10,737][44958] Updated weights for policy 0, policy_version 7720 (0.0008) [2023-10-12 20:16:11,106][44958] Updated weights for policy 0, policy_version 7730 (0.0010) [2023-10-12 20:16:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 15859712. Throughput: 0: 1651.8, 1: 1646.3. Samples: 3975508. Policy #0 lag: (min: 25.0, avg: 29.9, max: 56.0) [2023-10-12 20:16:11,443][43579] Avg episode reward: [(0, '252.100'), (1, '252.490')] [2023-10-12 20:16:11,444][44583] Saving new best policy, reward=252.490! [2023-10-12 20:16:11,485][44958] Updated weights for policy 0, policy_version 7740 (0.0009) [2023-10-12 20:16:14,302][44959] Updated weights for policy 1, policy_version 7780 (0.0007) [2023-10-12 20:16:14,675][44959] Updated weights for policy 1, policy_version 7790 (0.0008) [2023-10-12 20:16:15,050][44959] Updated weights for policy 1, policy_version 7800 (0.0008) [2023-10-12 20:16:15,550][44958] Updated weights for policy 0, policy_version 7750 (0.0007) [2023-10-12 20:16:15,926][44958] Updated weights for policy 0, policy_version 7760 (0.0008) [2023-10-12 20:16:16,295][44958] Updated weights for policy 0, policy_version 7770 (0.0008) [2023-10-12 20:16:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15925248. Throughput: 0: 1647.2, 1: 1648.2. Samples: 3994508. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 20:16:16,443][43579] Avg episode reward: [(0, '253.540'), (1, '248.950')] [2023-10-12 20:16:19,242][44959] Updated weights for policy 1, policy_version 7810 (0.0007) [2023-10-12 20:16:19,613][44959] Updated weights for policy 1, policy_version 7820 (0.0007) [2023-10-12 20:16:19,975][44959] Updated weights for policy 1, policy_version 7830 (0.0008) [2023-10-12 20:16:20,341][44959] Updated weights for policy 1, policy_version 7840 (0.0007) [2023-10-12 20:16:20,358][44958] Updated weights for policy 0, policy_version 7780 (0.0009) [2023-10-12 20:16:20,727][44958] Updated weights for policy 0, policy_version 7790 (0.0009) [2023-10-12 20:16:21,105][44958] Updated weights for policy 0, policy_version 7800 (0.0009) [2023-10-12 20:16:21,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 16023552. Throughput: 0: 1651.1, 1: 1651.1. Samples: 4005492. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:16:21,443][43579] Avg episode reward: [(0, '250.750'), (1, '244.670')] [2023-10-12 20:16:24,480][44959] Updated weights for policy 1, policy_version 7850 (0.0010) [2023-10-12 20:16:24,855][44959] Updated weights for policy 1, policy_version 7860 (0.0009) [2023-10-12 20:16:25,188][44958] Updated weights for policy 0, policy_version 7810 (0.0008) [2023-10-12 20:16:25,225][44959] Updated weights for policy 1, policy_version 7870 (0.0007) [2023-10-12 20:16:25,562][44958] Updated weights for policy 0, policy_version 7820 (0.0008) [2023-10-12 20:16:25,937][44958] Updated weights for policy 0, policy_version 7830 (0.0010) [2023-10-12 20:16:26,305][44958] Updated weights for policy 0, policy_version 7840 (0.0009) [2023-10-12 20:16:26,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 16089088. Throughput: 0: 1653.3, 1: 1643.3. Samples: 4024978. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:16:26,444][43579] Avg episode reward: [(0, '244.520'), (1, '242.380')] [2023-10-12 20:16:29,542][44959] Updated weights for policy 1, policy_version 7880 (0.0007) [2023-10-12 20:16:29,910][44959] Updated weights for policy 1, policy_version 7890 (0.0007) [2023-10-12 20:16:30,273][44959] Updated weights for policy 1, policy_version 7900 (0.0007) [2023-10-12 20:16:30,584][44958] Updated weights for policy 0, policy_version 7850 (0.0008) [2023-10-12 20:16:30,958][44958] Updated weights for policy 0, policy_version 7860 (0.0009) [2023-10-12 20:16:31,336][44958] Updated weights for policy 0, policy_version 7870 (0.0007) [2023-10-12 20:16:31,443][43579] Fps is (10 sec: 13106.5, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 16154624. Throughput: 0: 1649.5, 1: 1644.8. Samples: 4044024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:16:31,444][43579] Avg episode reward: [(0, '242.570'), (1, '246.990')] [2023-10-12 20:16:34,507][44959] Updated weights for policy 1, policy_version 7910 (0.0008) [2023-10-12 20:16:34,877][44959] Updated weights for policy 1, policy_version 7920 (0.0009) [2023-10-12 20:16:35,242][44959] Updated weights for policy 1, policy_version 7930 (0.0008) [2023-10-12 20:16:35,340][44958] Updated weights for policy 0, policy_version 7880 (0.0008) [2023-10-12 20:16:35,706][44958] Updated weights for policy 0, policy_version 7890 (0.0008) [2023-10-12 20:16:36,089][44958] Updated weights for policy 0, policy_version 7900 (0.0008) [2023-10-12 20:16:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 16220160. Throughput: 0: 1663.3, 1: 1645.1. Samples: 4055114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:16:36,443][43579] Avg episode reward: [(0, '246.950'), (1, '248.460')] [2023-10-12 20:16:39,422][44959] Updated weights for policy 1, policy_version 7940 (0.0008) [2023-10-12 20:16:39,785][44959] Updated weights for policy 1, policy_version 7950 (0.0010) [2023-10-12 20:16:40,157][44959] Updated weights for policy 1, policy_version 7960 (0.0009) [2023-10-12 20:16:40,218][44958] Updated weights for policy 0, policy_version 7910 (0.0009) [2023-10-12 20:16:40,596][44958] Updated weights for policy 0, policy_version 7920 (0.0007) [2023-10-12 20:16:40,962][44958] Updated weights for policy 0, policy_version 7930 (0.0009) [2023-10-12 20:16:41,443][43579] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 16285696. Throughput: 0: 1654.0, 1: 1641.0. Samples: 4074364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:16:41,444][43579] Avg episode reward: [(0, '246.830'), (1, '240.880')] [2023-10-12 20:16:44,321][44959] Updated weights for policy 1, policy_version 7970 (0.0007) [2023-10-12 20:16:44,699][44959] Updated weights for policy 1, policy_version 7980 (0.0008) [2023-10-12 20:16:45,077][44959] Updated weights for policy 1, policy_version 7990 (0.0009) [2023-10-12 20:16:45,095][44958] Updated weights for policy 0, policy_version 7940 (0.0008) [2023-10-12 20:16:45,443][44959] Updated weights for policy 1, policy_version 8000 (0.0008) [2023-10-12 20:16:45,491][44958] Updated weights for policy 0, policy_version 7950 (0.0008) [2023-10-12 20:16:45,867][44958] Updated weights for policy 0, policy_version 7960 (0.0007) [2023-10-12 20:16:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 16351232. Throughput: 0: 1646.5, 1: 1647.2. Samples: 4093128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:16:46,444][43579] Avg episode reward: [(0, '248.100'), (1, '240.440')] [2023-10-12 20:16:49,642][44959] Updated weights for policy 1, policy_version 8010 (0.0007) [2023-10-12 20:16:50,006][44959] Updated weights for policy 1, policy_version 8020 (0.0008) [2023-10-12 20:16:50,105][44958] Updated weights for policy 0, policy_version 7970 (0.0008) [2023-10-12 20:16:50,373][44959] Updated weights for policy 1, policy_version 8030 (0.0009) [2023-10-12 20:16:50,486][44958] Updated weights for policy 0, policy_version 7980 (0.0009) [2023-10-12 20:16:50,841][44958] Updated weights for policy 0, policy_version 7990 (0.0009) [2023-10-12 20:16:51,213][44958] Updated weights for policy 0, policy_version 8000 (0.0008) [2023-10-12 20:16:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 16416768. Throughput: 0: 1656.3, 1: 1643.9. Samples: 4104440. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-12 20:16:51,444][43579] Avg episode reward: [(0, '251.580'), (1, '237.400')] [2023-10-12 20:16:54,482][44959] Updated weights for policy 1, policy_version 8040 (0.0007) [2023-10-12 20:16:54,853][44959] Updated weights for policy 1, policy_version 8050 (0.0008) [2023-10-12 20:16:55,226][44959] Updated weights for policy 1, policy_version 8060 (0.0007) [2023-10-12 20:16:55,480][44958] Updated weights for policy 0, policy_version 8010 (0.0007) [2023-10-12 20:16:55,858][44958] Updated weights for policy 0, policy_version 8020 (0.0008) [2023-10-12 20:16:56,230][44958] Updated weights for policy 0, policy_version 8030 (0.0010) [2023-10-12 20:16:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 16482304. Throughput: 0: 1647.1, 1: 1643.3. Samples: 4123578. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-12 20:16:56,443][43579] Avg episode reward: [(0, '250.890'), (1, '244.390')] [2023-10-12 20:16:59,383][44959] Updated weights for policy 1, policy_version 8070 (0.0008) [2023-10-12 20:16:59,749][44959] Updated weights for policy 1, policy_version 8080 (0.0008) [2023-10-12 20:17:00,121][44959] Updated weights for policy 1, policy_version 8090 (0.0007) [2023-10-12 20:17:00,481][44958] Updated weights for policy 0, policy_version 8040 (0.0009) [2023-10-12 20:17:00,855][44958] Updated weights for policy 0, policy_version 8050 (0.0008) [2023-10-12 20:17:01,235][44958] Updated weights for policy 0, policy_version 8060 (0.0008) [2023-10-12 20:17:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 16547840. Throughput: 0: 1640.6, 1: 1645.6. Samples: 4142390. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 20:17:01,444][43579] Avg episode reward: [(0, '249.660'), (1, '245.720')] [2023-10-12 20:17:04,430][44959] Updated weights for policy 1, policy_version 8100 (0.0007) [2023-10-12 20:17:04,802][44959] Updated weights for policy 1, policy_version 8110 (0.0007) [2023-10-12 20:17:05,168][44959] Updated weights for policy 1, policy_version 8120 (0.0007) [2023-10-12 20:17:05,456][44958] Updated weights for policy 0, policy_version 8070 (0.0008) [2023-10-12 20:17:05,824][44958] Updated weights for policy 0, policy_version 8080 (0.0009) [2023-10-12 20:17:06,191][44958] Updated weights for policy 0, policy_version 8090 (0.0009) [2023-10-12 20:17:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 16613376. Throughput: 0: 1644.9, 1: 1639.9. Samples: 4153310. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 20:17:06,444][43579] Avg episode reward: [(0, '247.520'), (1, '245.550')] [2023-10-12 20:17:09,204][44959] Updated weights for policy 1, policy_version 8130 (0.0008) [2023-10-12 20:17:09,575][44959] Updated weights for policy 1, policy_version 8140 (0.0009) [2023-10-12 20:17:09,940][44959] Updated weights for policy 1, policy_version 8150 (0.0009) [2023-10-12 20:17:10,313][44959] Updated weights for policy 1, policy_version 8160 (0.0009) [2023-10-12 20:17:10,366][44958] Updated weights for policy 0, policy_version 8100 (0.0009) [2023-10-12 20:17:10,744][44958] Updated weights for policy 0, policy_version 8110 (0.0007) [2023-10-12 20:17:11,115][44958] Updated weights for policy 0, policy_version 8120 (0.0009) [2023-10-12 20:17:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 16678912. Throughput: 0: 1643.0, 1: 1638.8. Samples: 4172658. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-12 20:17:11,443][43579] Avg episode reward: [(0, '249.550'), (1, '246.130')] [2023-10-12 20:17:14,586][44959] Updated weights for policy 1, policy_version 8170 (0.0008) [2023-10-12 20:17:14,956][44959] Updated weights for policy 1, policy_version 8180 (0.0008) [2023-10-12 20:17:15,206][44958] Updated weights for policy 0, policy_version 8130 (0.0009) [2023-10-12 20:17:15,326][44959] Updated weights for policy 1, policy_version 8190 (0.0007) [2023-10-12 20:17:15,591][44958] Updated weights for policy 0, policy_version 8140 (0.0009) [2023-10-12 20:17:15,965][44958] Updated weights for policy 0, policy_version 8150 (0.0009) [2023-10-12 20:17:16,321][44958] Updated weights for policy 0, policy_version 8160 (0.0009) [2023-10-12 20:17:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 16744448. Throughput: 0: 1641.2, 1: 1636.5. Samples: 4191520. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-12 20:17:16,444][43579] Avg episode reward: [(0, '250.490'), (1, '245.090')] [2023-10-12 20:17:19,389][44959] Updated weights for policy 1, policy_version 8200 (0.0009) [2023-10-12 20:17:19,769][44959] Updated weights for policy 1, policy_version 8210 (0.0009) [2023-10-12 20:17:20,132][44959] Updated weights for policy 1, policy_version 8220 (0.0009) [2023-10-12 20:17:20,527][44958] Updated weights for policy 0, policy_version 8170 (0.0011) [2023-10-12 20:17:20,911][44958] Updated weights for policy 0, policy_version 8180 (0.0008) [2023-10-12 20:17:21,284][44958] Updated weights for policy 0, policy_version 8190 (0.0008) [2023-10-12 20:17:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 16809984. Throughput: 0: 1636.3, 1: 1644.7. Samples: 4202760. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 20:17:21,444][43579] Avg episode reward: [(0, '249.890'), (1, '243.720')] [2023-10-12 20:17:24,303][44959] Updated weights for policy 1, policy_version 8230 (0.0008) [2023-10-12 20:17:24,675][44959] Updated weights for policy 1, policy_version 8240 (0.0007) [2023-10-12 20:17:25,051][44959] Updated weights for policy 1, policy_version 8250 (0.0007) [2023-10-12 20:17:25,522][44958] Updated weights for policy 0, policy_version 8200 (0.0009) [2023-10-12 20:17:25,897][44958] Updated weights for policy 0, policy_version 8210 (0.0009) [2023-10-12 20:17:26,275][44958] Updated weights for policy 0, policy_version 8220 (0.0009) [2023-10-12 20:17:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16875520. Throughput: 0: 1637.5, 1: 1642.9. Samples: 4221984. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 20:17:26,443][43579] Avg episode reward: [(0, '254.770'), (1, '241.860')] [2023-10-12 20:17:29,186][44959] Updated weights for policy 1, policy_version 8260 (0.0008) [2023-10-12 20:17:29,559][44959] Updated weights for policy 1, policy_version 8270 (0.0008) [2023-10-12 20:17:29,934][44959] Updated weights for policy 1, policy_version 8280 (0.0008) [2023-10-12 20:17:30,644][44958] Updated weights for policy 0, policy_version 8230 (0.0008) [2023-10-12 20:17:31,025][44958] Updated weights for policy 0, policy_version 8240 (0.0009) [2023-10-12 20:17:31,406][44958] Updated weights for policy 0, policy_version 8250 (0.0011) [2023-10-12 20:17:31,442][43579] Fps is (10 sec: 9830.6, 60 sec: 12561.2, 300 sec: 13218.3). Total num frames: 16908288. Throughput: 0: 1642.3, 1: 1644.5. Samples: 4241034. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 20:17:31,443][43579] Avg episode reward: [(0, '255.000'), (1, '246.730')] [2023-10-12 20:17:33,989][44959] Updated weights for policy 1, policy_version 8290 (0.0009) [2023-10-12 20:17:34,350][44959] Updated weights for policy 1, policy_version 8300 (0.0009) [2023-10-12 20:17:34,726][44959] Updated weights for policy 1, policy_version 8310 (0.0009) [2023-10-12 20:17:35,087][44959] Updated weights for policy 1, policy_version 8320 (0.0008) [2023-10-12 20:17:35,339][44958] Updated weights for policy 0, policy_version 8260 (0.0009) [2023-10-12 20:17:35,717][44958] Updated weights for policy 0, policy_version 8270 (0.0007) [2023-10-12 20:17:36,088][44958] Updated weights for policy 0, policy_version 8280 (0.0008) [2023-10-12 20:17:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17006592. Throughput: 0: 1625.8, 1: 1643.2. Samples: 4251542. Policy #0 lag: (min: 30.0, avg: 31.8, max: 59.0) [2023-10-12 20:17:36,443][43579] Avg episode reward: [(0, '258.310'), (1, '243.170')] [2023-10-12 20:17:36,444][44518] Saving new best policy, reward=258.310! [2023-10-12 20:17:39,204][44959] Updated weights for policy 1, policy_version 8330 (0.0009) [2023-10-12 20:17:39,578][44959] Updated weights for policy 1, policy_version 8340 (0.0009) [2023-10-12 20:17:39,952][44959] Updated weights for policy 1, policy_version 8350 (0.0010) [2023-10-12 20:17:40,238][44958] Updated weights for policy 0, policy_version 8290 (0.0009) [2023-10-12 20:17:40,620][44958] Updated weights for policy 0, policy_version 8300 (0.0009) [2023-10-12 20:17:40,987][44958] Updated weights for policy 0, policy_version 8310 (0.0007) [2023-10-12 20:17:41,364][44958] Updated weights for policy 0, policy_version 8320 (0.0007) [2023-10-12 20:17:41,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17072128. Throughput: 0: 1632.4, 1: 1640.9. Samples: 4270878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:17:41,444][43579] Avg episode reward: [(0, '260.710'), (1, '248.910')] [2023-10-12 20:17:41,445][44518] Saving new best policy, reward=260.710! [2023-10-12 20:17:44,140][44959] Updated weights for policy 1, policy_version 8360 (0.0009) [2023-10-12 20:17:44,521][44959] Updated weights for policy 1, policy_version 8370 (0.0007) [2023-10-12 20:17:44,896][44959] Updated weights for policy 1, policy_version 8380 (0.0008) [2023-10-12 20:17:45,558][44958] Updated weights for policy 0, policy_version 8330 (0.0008) [2023-10-12 20:17:45,932][44958] Updated weights for policy 0, policy_version 8340 (0.0008) [2023-10-12 20:17:46,304][44958] Updated weights for policy 0, policy_version 8350 (0.0008) [2023-10-12 20:17:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 17137664. Throughput: 0: 1635.7, 1: 1647.8. Samples: 4290150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:17:46,445][43579] Avg episode reward: [(0, '258.300'), (1, '255.610')] [2023-10-12 20:17:46,457][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000008384_8585216.pth... [2023-10-12 20:17:46,458][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000008352_8552448.pth... [2023-10-12 20:17:46,493][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000006848_7012352.pth [2023-10-12 20:17:46,494][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000006816_6979584.pth [2023-10-12 20:17:46,497][44583] Saving new best policy, reward=255.610! [2023-10-12 20:17:49,170][44959] Updated weights for policy 1, policy_version 8390 (0.0009) [2023-10-12 20:17:49,541][44959] Updated weights for policy 1, policy_version 8400 (0.0009) [2023-10-12 20:17:49,912][44959] Updated weights for policy 1, policy_version 8410 (0.0009) [2023-10-12 20:17:50,457][44958] Updated weights for policy 0, policy_version 8360 (0.0010) [2023-10-12 20:17:50,821][44958] Updated weights for policy 0, policy_version 8370 (0.0010) [2023-10-12 20:17:51,198][44958] Updated weights for policy 0, policy_version 8380 (0.0010) [2023-10-12 20:17:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17203200. Throughput: 0: 1635.8, 1: 1644.1. Samples: 4300904. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-12 20:17:51,444][43579] Avg episode reward: [(0, '259.650'), (1, '258.980')] [2023-10-12 20:17:51,445][44583] Saving new best policy, reward=258.980! [2023-10-12 20:17:54,220][44959] Updated weights for policy 1, policy_version 8420 (0.0009) [2023-10-12 20:17:54,598][44959] Updated weights for policy 1, policy_version 8430 (0.0010) [2023-10-12 20:17:54,970][44959] Updated weights for policy 1, policy_version 8440 (0.0009) [2023-10-12 20:17:55,415][44958] Updated weights for policy 0, policy_version 8390 (0.0008) [2023-10-12 20:17:55,785][44958] Updated weights for policy 0, policy_version 8400 (0.0008) [2023-10-12 20:17:56,144][44958] Updated weights for policy 0, policy_version 8410 (0.0009) [2023-10-12 20:17:56,443][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 17268736. Throughput: 0: 1634.7, 1: 1645.5. Samples: 4320266. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-12 20:17:56,443][43579] Avg episode reward: [(0, '262.670'), (1, '255.080')] [2023-10-12 20:17:56,444][44518] Saving new best policy, reward=262.670! [2023-10-12 20:17:59,052][44959] Updated weights for policy 1, policy_version 8450 (0.0009) [2023-10-12 20:17:59,422][44959] Updated weights for policy 1, policy_version 8460 (0.0009) [2023-10-12 20:17:59,790][44959] Updated weights for policy 1, policy_version 8470 (0.0009) [2023-10-12 20:18:00,157][44959] Updated weights for policy 1, policy_version 8480 (0.0008) [2023-10-12 20:18:00,431][44958] Updated weights for policy 0, policy_version 8420 (0.0010) [2023-10-12 20:18:00,803][44958] Updated weights for policy 0, policy_version 8430 (0.0009) [2023-10-12 20:18:01,178][44958] Updated weights for policy 0, policy_version 8440 (0.0009) [2023-10-12 20:18:01,443][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 17301504. Throughput: 0: 1632.9, 1: 1651.0. Samples: 4339296. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-12 20:18:01,444][43579] Avg episode reward: [(0, '262.600'), (1, '257.140')] [2023-10-12 20:18:04,211][44959] Updated weights for policy 1, policy_version 8490 (0.0007) [2023-10-12 20:18:04,585][44959] Updated weights for policy 1, policy_version 8500 (0.0007) [2023-10-12 20:18:04,960][44959] Updated weights for policy 1, policy_version 8510 (0.0009) [2023-10-12 20:18:05,367][44958] Updated weights for policy 0, policy_version 8450 (0.0007) [2023-10-12 20:18:05,734][44958] Updated weights for policy 0, policy_version 8460 (0.0007) [2023-10-12 20:18:06,105][44958] Updated weights for policy 0, policy_version 8470 (0.0009) [2023-10-12 20:18:06,443][43579] Fps is (10 sec: 9830.3, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 17367040. Throughput: 0: 1629.8, 1: 1642.4. Samples: 4350008. Policy #0 lag: (min: 21.0, avg: 25.0, max: 53.0) [2023-10-12 20:18:06,443][43579] Avg episode reward: [(0, '258.440'), (1, '253.190')] [2023-10-12 20:18:06,483][44958] Updated weights for policy 0, policy_version 8480 (0.0007) [2023-10-12 20:18:09,193][44959] Updated weights for policy 1, policy_version 8520 (0.0008) [2023-10-12 20:18:09,557][44959] Updated weights for policy 1, policy_version 8530 (0.0008) [2023-10-12 20:18:09,920][44959] Updated weights for policy 1, policy_version 8540 (0.0009) [2023-10-12 20:18:10,510][44958] Updated weights for policy 0, policy_version 8490 (0.0009) [2023-10-12 20:18:10,885][44958] Updated weights for policy 0, policy_version 8500 (0.0007) [2023-10-12 20:18:11,254][44958] Updated weights for policy 0, policy_version 8510 (0.0009) [2023-10-12 20:18:11,443][43579] Fps is (10 sec: 16384.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17465344. Throughput: 0: 1634.2, 1: 1641.7. Samples: 4369400. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-12 20:18:11,443][43579] Avg episode reward: [(0, '259.710'), (1, '247.610')] [2023-10-12 20:18:14,077][44959] Updated weights for policy 1, policy_version 8550 (0.0007) [2023-10-12 20:18:14,440][44959] Updated weights for policy 1, policy_version 8560 (0.0007) [2023-10-12 20:18:14,812][44959] Updated weights for policy 1, policy_version 8570 (0.0007) [2023-10-12 20:18:15,659][44958] Updated weights for policy 0, policy_version 8520 (0.0009) [2023-10-12 20:18:16,040][44958] Updated weights for policy 0, policy_version 8530 (0.0007) [2023-10-12 20:18:16,409][44958] Updated weights for policy 0, policy_version 8540 (0.0007) [2023-10-12 20:18:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 17498112. Throughput: 0: 1630.5, 1: 1646.1. Samples: 4388482. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-12 20:18:16,443][43579] Avg episode reward: [(0, '262.140'), (1, '243.010')] [2023-10-12 20:18:18,687][44959] Updated weights for policy 1, policy_version 8580 (0.0008) [2023-10-12 20:18:19,060][44959] Updated weights for policy 1, policy_version 8590 (0.0007) [2023-10-12 20:18:19,428][44959] Updated weights for policy 1, policy_version 8600 (0.0007) [2023-10-12 20:18:20,577][44958] Updated weights for policy 0, policy_version 8550 (0.0007) [2023-10-12 20:18:20,948][44958] Updated weights for policy 0, policy_version 8560 (0.0007) [2023-10-12 20:18:21,328][44958] Updated weights for policy 0, policy_version 8570 (0.0009) [2023-10-12 20:18:21,442][43579] Fps is (10 sec: 9830.5, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 17563648. Throughput: 0: 1638.6, 1: 1639.5. Samples: 4399058. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-12 20:18:21,443][43579] Avg episode reward: [(0, '257.370'), (1, '240.560')] [2023-10-12 20:18:23,528][44959] Updated weights for policy 1, policy_version 8610 (0.0009) [2023-10-12 20:18:23,895][44959] Updated weights for policy 1, policy_version 8620 (0.0010) [2023-10-12 20:18:24,258][44959] Updated weights for policy 1, policy_version 8630 (0.0007) [2023-10-12 20:18:24,635][44959] Updated weights for policy 1, policy_version 8640 (0.0008) [2023-10-12 20:18:25,343][44958] Updated weights for policy 0, policy_version 8580 (0.0007) [2023-10-12 20:18:25,715][44958] Updated weights for policy 0, policy_version 8590 (0.0007) [2023-10-12 20:18:26,080][44958] Updated weights for policy 0, policy_version 8600 (0.0008) [2023-10-12 20:18:26,442][43579] Fps is (10 sec: 16383.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17661952. Throughput: 0: 1637.0, 1: 1648.1. Samples: 4418710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:18:26,443][43579] Avg episode reward: [(0, '256.340'), (1, '243.500')] [2023-10-12 20:18:29,113][44959] Updated weights for policy 1, policy_version 8650 (0.0010) [2023-10-12 20:18:29,492][44959] Updated weights for policy 1, policy_version 8660 (0.0007) [2023-10-12 20:18:29,867][44959] Updated weights for policy 1, policy_version 8670 (0.0008) [2023-10-12 20:18:30,406][44958] Updated weights for policy 0, policy_version 8610 (0.0010) [2023-10-12 20:18:30,780][44958] Updated weights for policy 0, policy_version 8620 (0.0009) [2023-10-12 20:18:31,159][44958] Updated weights for policy 0, policy_version 8630 (0.0007) [2023-10-12 20:18:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 17694720. Throughput: 0: 1637.2, 1: 1645.4. Samples: 4437866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:18:31,443][43579] Avg episode reward: [(0, '253.960'), (1, '241.280')] [2023-10-12 20:18:31,525][44958] Updated weights for policy 0, policy_version 8640 (0.0009) [2023-10-12 20:18:34,014][44959] Updated weights for policy 1, policy_version 8680 (0.0009) [2023-10-12 20:18:34,382][44959] Updated weights for policy 1, policy_version 8690 (0.0008) [2023-10-12 20:18:34,742][44959] Updated weights for policy 1, policy_version 8700 (0.0007) [2023-10-12 20:18:35,734][44958] Updated weights for policy 0, policy_version 8650 (0.0009) [2023-10-12 20:18:36,112][44958] Updated weights for policy 0, policy_version 8660 (0.0009) [2023-10-12 20:18:36,442][43579] Fps is (10 sec: 9830.5, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 17760256. Throughput: 0: 1631.7, 1: 1645.3. Samples: 4448366. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-12 20:18:36,443][43579] Avg episode reward: [(0, '257.370'), (1, '239.250')] [2023-10-12 20:18:36,486][44958] Updated weights for policy 0, policy_version 8670 (0.0010) [2023-10-12 20:18:38,707][44959] Updated weights for policy 1, policy_version 8710 (0.0009) [2023-10-12 20:18:39,075][44959] Updated weights for policy 1, policy_version 8720 (0.0010) [2023-10-12 20:18:39,447][44959] Updated weights for policy 1, policy_version 8730 (0.0008) [2023-10-12 20:18:40,644][44958] Updated weights for policy 0, policy_version 8680 (0.0009) [2023-10-12 20:18:41,024][44958] Updated weights for policy 0, policy_version 8690 (0.0008) [2023-10-12 20:18:41,396][44958] Updated weights for policy 0, policy_version 8700 (0.0008) [2023-10-12 20:18:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 17825792. Throughput: 0: 1634.5, 1: 1651.9. Samples: 4468154. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-12 20:18:41,443][43579] Avg episode reward: [(0, '256.190'), (1, '241.000')] [2023-10-12 20:18:43,642][44959] Updated weights for policy 1, policy_version 8740 (0.0009) [2023-10-12 20:18:44,019][44959] Updated weights for policy 1, policy_version 8750 (0.0008) [2023-10-12 20:18:44,378][44959] Updated weights for policy 1, policy_version 8760 (0.0009) [2023-10-12 20:18:45,422][44958] Updated weights for policy 0, policy_version 8710 (0.0009) [2023-10-12 20:18:45,790][44958] Updated weights for policy 0, policy_version 8720 (0.0010) [2023-10-12 20:18:46,161][44958] Updated weights for policy 0, policy_version 8730 (0.0010) [2023-10-12 20:18:46,443][43579] Fps is (10 sec: 16383.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 17924096. Throughput: 0: 1633.5, 1: 1659.1. Samples: 4487462. Policy #0 lag: (min: 15.0, avg: 18.9, max: 47.0) [2023-10-12 20:18:46,444][43579] Avg episode reward: [(0, '253.770'), (1, '235.440')] [2023-10-12 20:18:48,533][44959] Updated weights for policy 1, policy_version 8770 (0.0011) [2023-10-12 20:18:48,903][44959] Updated weights for policy 1, policy_version 8780 (0.0008) [2023-10-12 20:18:49,269][44959] Updated weights for policy 1, policy_version 8790 (0.0007) [2023-10-12 20:18:49,640][44959] Updated weights for policy 1, policy_version 8800 (0.0008) [2023-10-12 20:18:50,481][44958] Updated weights for policy 0, policy_version 8740 (0.0008) [2023-10-12 20:18:50,864][44958] Updated weights for policy 0, policy_version 8750 (0.0008) [2023-10-12 20:18:51,234][44958] Updated weights for policy 0, policy_version 8760 (0.0007) [2023-10-12 20:18:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 17956864. Throughput: 0: 1642.8, 1: 1644.6. Samples: 4497938. Policy #0 lag: (min: 15.0, avg: 18.9, max: 47.0) [2023-10-12 20:18:51,443][43579] Avg episode reward: [(0, '259.190'), (1, '231.310')] [2023-10-12 20:18:53,629][44959] Updated weights for policy 1, policy_version 8810 (0.0010) [2023-10-12 20:18:53,992][44959] Updated weights for policy 1, policy_version 8820 (0.0009) [2023-10-12 20:18:54,364][44959] Updated weights for policy 1, policy_version 8830 (0.0007) [2023-10-12 20:18:55,479][44958] Updated weights for policy 0, policy_version 8770 (0.0008) [2023-10-12 20:18:55,862][44958] Updated weights for policy 0, policy_version 8780 (0.0007) [2023-10-12 20:18:56,235][44958] Updated weights for policy 0, policy_version 8790 (0.0007) [2023-10-12 20:18:56,442][43579] Fps is (10 sec: 9830.7, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 18022400. Throughput: 0: 1637.5, 1: 1657.4. Samples: 4517670. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-12 20:18:56,443][43579] Avg episode reward: [(0, '260.340'), (1, '230.740')] [2023-10-12 20:18:56,606][44958] Updated weights for policy 0, policy_version 8800 (0.0008) [2023-10-12 20:18:58,541][44959] Updated weights for policy 1, policy_version 8840 (0.0007) [2023-10-12 20:18:58,909][44959] Updated weights for policy 1, policy_version 8850 (0.0007) [2023-10-12 20:18:59,275][44959] Updated weights for policy 1, policy_version 8860 (0.0007) [2023-10-12 20:19:00,803][44958] Updated weights for policy 0, policy_version 8810 (0.0007) [2023-10-12 20:19:01,176][44958] Updated weights for policy 0, policy_version 8820 (0.0007) [2023-10-12 20:19:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18087936. Throughput: 0: 1640.2, 1: 1662.7. Samples: 4537112. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-12 20:19:01,443][43579] Avg episode reward: [(0, '259.620'), (1, '238.130')] [2023-10-12 20:19:01,545][44958] Updated weights for policy 0, policy_version 8830 (0.0007) [2023-10-12 20:19:03,354][44959] Updated weights for policy 1, policy_version 8870 (0.0008) [2023-10-12 20:19:03,718][44959] Updated weights for policy 1, policy_version 8880 (0.0008) [2023-10-12 20:19:04,090][44959] Updated weights for policy 1, policy_version 8890 (0.0008) [2023-10-12 20:19:05,510][44958] Updated weights for policy 0, policy_version 8840 (0.0008) [2023-10-12 20:19:05,887][44958] Updated weights for policy 0, policy_version 8850 (0.0009) [2023-10-12 20:19:06,251][44958] Updated weights for policy 0, policy_version 8860 (0.0009) [2023-10-12 20:19:06,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 18186240. Throughput: 0: 1641.8, 1: 1650.7. Samples: 4547218. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:19:06,443][43579] Avg episode reward: [(0, '260.130'), (1, '224.880')] [2023-10-12 20:19:08,266][44959] Updated weights for policy 1, policy_version 8900 (0.0008) [2023-10-12 20:19:08,634][44959] Updated weights for policy 1, policy_version 8910 (0.0007) [2023-10-12 20:19:09,014][44959] Updated weights for policy 1, policy_version 8920 (0.0007) [2023-10-12 20:19:10,446][44958] Updated weights for policy 0, policy_version 8870 (0.0010) [2023-10-12 20:19:10,814][44958] Updated weights for policy 0, policy_version 8880 (0.0008) [2023-10-12 20:19:11,189][44958] Updated weights for policy 0, policy_version 8890 (0.0010) [2023-10-12 20:19:11,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 18251776. Throughput: 0: 1641.8, 1: 1661.1. Samples: 4567340. Policy #0 lag: (min: 17.0, avg: 22.7, max: 49.0) [2023-10-12 20:19:11,444][43579] Avg episode reward: [(0, '258.560'), (1, '229.030')] [2023-10-12 20:19:12,971][44959] Updated weights for policy 1, policy_version 8930 (0.0009) [2023-10-12 20:19:13,382][44959] Updated weights for policy 1, policy_version 8940 (0.0008) [2023-10-12 20:19:13,752][44959] Updated weights for policy 1, policy_version 8950 (0.0008) [2023-10-12 20:19:14,126][44959] Updated weights for policy 1, policy_version 8960 (0.0007) [2023-10-12 20:19:15,444][44958] Updated weights for policy 0, policy_version 8900 (0.0009) [2023-10-12 20:19:15,830][44958] Updated weights for policy 0, policy_version 8910 (0.0007) [2023-10-12 20:19:16,199][44958] Updated weights for policy 0, policy_version 8920 (0.0008) [2023-10-12 20:19:16,443][43579] Fps is (10 sec: 9830.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 18284544. Throughput: 0: 1640.7, 1: 1672.5. Samples: 4586964. Policy #0 lag: (min: 17.0, avg: 22.7, max: 49.0) [2023-10-12 20:19:16,444][43579] Avg episode reward: [(0, '259.250'), (1, '231.470')] [2023-10-12 20:19:18,188][44959] Updated weights for policy 1, policy_version 8970 (0.0010) [2023-10-12 20:19:18,561][44959] Updated weights for policy 1, policy_version 8980 (0.0008) [2023-10-12 20:19:18,922][44959] Updated weights for policy 1, policy_version 8990 (0.0009) [2023-10-12 20:19:20,322][44958] Updated weights for policy 0, policy_version 8930 (0.0008) [2023-10-12 20:19:20,685][44958] Updated weights for policy 0, policy_version 8940 (0.0008) [2023-10-12 20:19:21,061][44958] Updated weights for policy 0, policy_version 8950 (0.0007) [2023-10-12 20:19:21,434][44958] Updated weights for policy 0, policy_version 8960 (0.0007) [2023-10-12 20:19:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 18382848. Throughput: 0: 1644.5, 1: 1650.1. Samples: 4596622. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-12 20:19:21,444][43579] Avg episode reward: [(0, '258.490'), (1, '228.040')] [2023-10-12 20:19:23,098][44959] Updated weights for policy 1, policy_version 9000 (0.0010) [2023-10-12 20:19:23,468][44959] Updated weights for policy 1, policy_version 9010 (0.0008) [2023-10-12 20:19:23,834][44959] Updated weights for policy 1, policy_version 9020 (0.0009) [2023-10-12 20:19:25,604][44958] Updated weights for policy 0, policy_version 8970 (0.0007) [2023-10-12 20:19:25,988][44958] Updated weights for policy 0, policy_version 8980 (0.0010) [2023-10-12 20:19:26,366][44958] Updated weights for policy 0, policy_version 8990 (0.0008) [2023-10-12 20:19:26,442][43579] Fps is (10 sec: 16384.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 18448384. Throughput: 0: 1639.6, 1: 1667.6. Samples: 4616976. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-12 20:19:26,443][43579] Avg episode reward: [(0, '259.880'), (1, '232.430')] [2023-10-12 20:19:28,008][44959] Updated weights for policy 1, policy_version 9030 (0.0010) [2023-10-12 20:19:28,384][44959] Updated weights for policy 1, policy_version 9040 (0.0008) [2023-10-12 20:19:28,749][44959] Updated weights for policy 1, policy_version 9050 (0.0009) [2023-10-12 20:19:30,605][44958] Updated weights for policy 0, policy_version 9000 (0.0008) [2023-10-12 20:19:30,969][44958] Updated weights for policy 0, policy_version 9010 (0.0009) [2023-10-12 20:19:31,340][44958] Updated weights for policy 0, policy_version 9020 (0.0009) [2023-10-12 20:19:31,443][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 18481152. Throughput: 0: 1640.4, 1: 1665.6. Samples: 4636232. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-12 20:19:31,444][43579] Avg episode reward: [(0, '259.920'), (1, '242.400')] [2023-10-12 20:19:32,891][44959] Updated weights for policy 1, policy_version 9060 (0.0010) [2023-10-12 20:19:33,264][44959] Updated weights for policy 1, policy_version 9070 (0.0010) [2023-10-12 20:19:33,625][44959] Updated weights for policy 1, policy_version 9080 (0.0007) [2023-10-12 20:19:35,378][44958] Updated weights for policy 0, policy_version 9030 (0.0008) [2023-10-12 20:19:35,749][44958] Updated weights for policy 0, policy_version 9040 (0.0009) [2023-10-12 20:19:36,124][44958] Updated weights for policy 0, policy_version 9050 (0.0008) [2023-10-12 20:19:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 18579456. Throughput: 0: 1633.5, 1: 1655.6. Samples: 4645944. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:19:36,443][43579] Avg episode reward: [(0, '258.190'), (1, '244.420')] [2023-10-12 20:19:37,765][44959] Updated weights for policy 1, policy_version 9090 (0.0007) [2023-10-12 20:19:38,138][44959] Updated weights for policy 1, policy_version 9100 (0.0009) [2023-10-12 20:19:38,504][44959] Updated weights for policy 1, policy_version 9110 (0.0008) [2023-10-12 20:19:38,873][44959] Updated weights for policy 1, policy_version 9120 (0.0008) [2023-10-12 20:19:40,496][44958] Updated weights for policy 0, policy_version 9060 (0.0010) [2023-10-12 20:19:40,868][44958] Updated weights for policy 0, policy_version 9070 (0.0010) [2023-10-12 20:19:41,243][44958] Updated weights for policy 0, policy_version 9080 (0.0011) [2023-10-12 20:19:41,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18612224. Throughput: 0: 1629.5, 1: 1665.6. Samples: 4665946. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:19:41,443][43579] Avg episode reward: [(0, '263.840'), (1, '250.280')] [2023-10-12 20:19:41,542][44518] Saving new best policy, reward=263.840! [2023-10-12 20:19:43,051][44959] Updated weights for policy 1, policy_version 9130 (0.0010) [2023-10-12 20:19:43,417][44959] Updated weights for policy 1, policy_version 9140 (0.0008) [2023-10-12 20:19:43,791][44959] Updated weights for policy 1, policy_version 9150 (0.0008) [2023-10-12 20:19:45,411][44958] Updated weights for policy 0, policy_version 9090 (0.0009) [2023-10-12 20:19:45,795][44958] Updated weights for policy 0, policy_version 9100 (0.0007) [2023-10-12 20:19:46,174][44958] Updated weights for policy 0, policy_version 9110 (0.0008) [2023-10-12 20:19:46,443][43579] Fps is (10 sec: 9830.1, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 18677760. Throughput: 0: 1636.1, 1: 1669.9. Samples: 4685880. Policy #0 lag: (min: 26.0, avg: 33.5, max: 58.0) [2023-10-12 20:19:46,444][43579] Avg episode reward: [(0, '263.540'), (1, '240.120')] [2023-10-12 20:19:46,456][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000009152_9371648.pth... [2023-10-12 20:19:46,491][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000007616_7798784.pth [2023-10-12 20:19:46,534][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000009120_9338880.pth... [2023-10-12 20:19:46,538][44958] Updated weights for policy 0, policy_version 9120 (0.0009) [2023-10-12 20:19:46,567][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000007584_7766016.pth [2023-10-12 20:19:47,786][44959] Updated weights for policy 1, policy_version 9160 (0.0009) [2023-10-12 20:19:48,159][44959] Updated weights for policy 1, policy_version 9170 (0.0010) [2023-10-12 20:19:48,528][44959] Updated weights for policy 1, policy_version 9180 (0.0010) [2023-10-12 20:19:50,762][44958] Updated weights for policy 0, policy_version 9130 (0.0010) [2023-10-12 20:19:51,123][44958] Updated weights for policy 0, policy_version 9140 (0.0008) [2023-10-12 20:19:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18743296. Throughput: 0: 1630.9, 1: 1662.1. Samples: 4695404. Policy #0 lag: (min: 26.0, avg: 33.5, max: 58.0) [2023-10-12 20:19:51,443][43579] Avg episode reward: [(0, '266.440'), (1, '247.370')] [2023-10-12 20:19:51,490][44958] Updated weights for policy 0, policy_version 9150 (0.0008) [2023-10-12 20:19:51,565][44518] Saving new best policy, reward=266.440! [2023-10-12 20:19:52,653][44959] Updated weights for policy 1, policy_version 9190 (0.0009) [2023-10-12 20:19:53,029][44959] Updated weights for policy 1, policy_version 9200 (0.0009) [2023-10-12 20:19:53,397][44959] Updated weights for policy 1, policy_version 9210 (0.0008) [2023-10-12 20:19:55,850][44958] Updated weights for policy 0, policy_version 9160 (0.0007) [2023-10-12 20:19:56,220][44958] Updated weights for policy 0, policy_version 9170 (0.0008) [2023-10-12 20:19:56,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18808832. Throughput: 0: 1622.0, 1: 1664.3. Samples: 4715224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:19:56,443][43579] Avg episode reward: [(0, '264.580'), (1, '241.730')] [2023-10-12 20:19:56,600][44958] Updated weights for policy 0, policy_version 9180 (0.0008) [2023-10-12 20:19:57,539][44959] Updated weights for policy 1, policy_version 9220 (0.0009) [2023-10-12 20:19:57,911][44959] Updated weights for policy 1, policy_version 9230 (0.0011) [2023-10-12 20:19:58,279][44959] Updated weights for policy 1, policy_version 9240 (0.0011) [2023-10-12 20:20:00,691][44958] Updated weights for policy 0, policy_version 9190 (0.0008) [2023-10-12 20:20:01,060][44958] Updated weights for policy 0, policy_version 9200 (0.0008) [2023-10-12 20:20:01,438][44958] Updated weights for policy 0, policy_version 9210 (0.0007) [2023-10-12 20:20:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 18874368. Throughput: 0: 1629.7, 1: 1655.0. Samples: 4734774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:20:01,443][43579] Avg episode reward: [(0, '263.890'), (1, '250.700')] [2023-10-12 20:20:02,654][44959] Updated weights for policy 1, policy_version 9250 (0.0010) [2023-10-12 20:20:03,077][44959] Updated weights for policy 1, policy_version 9260 (0.0009) [2023-10-12 20:20:03,450][44959] Updated weights for policy 1, policy_version 9270 (0.0009) [2023-10-12 20:20:03,817][44959] Updated weights for policy 1, policy_version 9280 (0.0008) [2023-10-12 20:20:05,713][44958] Updated weights for policy 0, policy_version 9220 (0.0008) [2023-10-12 20:20:06,088][44958] Updated weights for policy 0, policy_version 9230 (0.0009) [2023-10-12 20:20:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 18939904. Throughput: 0: 1622.6, 1: 1656.3. Samples: 4744172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:20:06,444][43579] Avg episode reward: [(0, '257.570'), (1, '242.400')] [2023-10-12 20:20:06,462][44958] Updated weights for policy 0, policy_version 9240 (0.0011) [2023-10-12 20:20:07,830][44959] Updated weights for policy 1, policy_version 9290 (0.0008) [2023-10-12 20:20:08,203][44959] Updated weights for policy 1, policy_version 9300 (0.0009) [2023-10-12 20:20:08,584][44959] Updated weights for policy 1, policy_version 9310 (0.0007) [2023-10-12 20:20:10,520][44958] Updated weights for policy 0, policy_version 9250 (0.0010) [2023-10-12 20:20:10,888][44958] Updated weights for policy 0, policy_version 9260 (0.0009) [2023-10-12 20:20:11,259][44958] Updated weights for policy 0, policy_version 9270 (0.0010) [2023-10-12 20:20:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 19005440. Throughput: 0: 1625.6, 1: 1654.0. Samples: 4764556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:20:11,443][43579] Avg episode reward: [(0, '257.590'), (1, '242.730')] [2023-10-12 20:20:11,639][44958] Updated weights for policy 0, policy_version 9280 (0.0010) [2023-10-12 20:20:12,624][44959] Updated weights for policy 1, policy_version 9320 (0.0007) [2023-10-12 20:20:12,985][44959] Updated weights for policy 1, policy_version 9330 (0.0009) [2023-10-12 20:20:13,354][44959] Updated weights for policy 1, policy_version 9340 (0.0007) [2023-10-12 20:20:15,695][44958] Updated weights for policy 0, policy_version 9290 (0.0007) [2023-10-12 20:20:16,077][44958] Updated weights for policy 0, policy_version 9300 (0.0008) [2023-10-12 20:20:16,442][44958] Updated weights for policy 0, policy_version 9310 (0.0009) [2023-10-12 20:20:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 19070976. Throughput: 0: 1632.3, 1: 1657.3. Samples: 4784264. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-12 20:20:16,443][43579] Avg episode reward: [(0, '253.830'), (1, '245.480')] [2023-10-12 20:20:17,617][44959] Updated weights for policy 1, policy_version 9350 (0.0007) [2023-10-12 20:20:17,989][44959] Updated weights for policy 1, policy_version 9360 (0.0008) [2023-10-12 20:20:18,362][44959] Updated weights for policy 1, policy_version 9370 (0.0009) [2023-10-12 20:20:20,903][44958] Updated weights for policy 0, policy_version 9320 (0.0009) [2023-10-12 20:20:21,271][44958] Updated weights for policy 0, policy_version 9330 (0.0009) [2023-10-12 20:20:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 19136512. Throughput: 0: 1630.1, 1: 1656.6. Samples: 4793846. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-12 20:20:21,443][43579] Avg episode reward: [(0, '248.840'), (1, '245.470')] [2023-10-12 20:20:21,645][44958] Updated weights for policy 0, policy_version 9340 (0.0009) [2023-10-12 20:20:22,456][44959] Updated weights for policy 1, policy_version 9380 (0.0009) [2023-10-12 20:20:22,823][44959] Updated weights for policy 1, policy_version 9390 (0.0007) [2023-10-12 20:20:23,198][44959] Updated weights for policy 1, policy_version 9400 (0.0007) [2023-10-12 20:20:25,813][44958] Updated weights for policy 0, policy_version 9350 (0.0007) [2023-10-12 20:20:26,189][44958] Updated weights for policy 0, policy_version 9360 (0.0008) [2023-10-12 20:20:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 19202048. Throughput: 0: 1637.6, 1: 1654.9. Samples: 4814110. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-12 20:20:26,443][43579] Avg episode reward: [(0, '248.460'), (1, '240.140')] [2023-10-12 20:20:26,558][44958] Updated weights for policy 0, policy_version 9370 (0.0010) [2023-10-12 20:20:27,295][44959] Updated weights for policy 1, policy_version 9410 (0.0008) [2023-10-12 20:20:27,665][44959] Updated weights for policy 1, policy_version 9420 (0.0010) [2023-10-12 20:20:28,041][44959] Updated weights for policy 1, policy_version 9430 (0.0009) [2023-10-12 20:20:28,395][44959] Updated weights for policy 1, policy_version 9440 (0.0009) [2023-10-12 20:20:30,791][44958] Updated weights for policy 0, policy_version 9380 (0.0010) [2023-10-12 20:20:31,190][44958] Updated weights for policy 0, policy_version 9390 (0.0011) [2023-10-12 20:20:31,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19267584. Throughput: 0: 1633.6, 1: 1651.2. Samples: 4833700. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-12 20:20:31,444][43579] Avg episode reward: [(0, '250.040'), (1, '247.360')] [2023-10-12 20:20:31,555][44958] Updated weights for policy 0, policy_version 9400 (0.0009) [2023-10-12 20:20:32,612][44959] Updated weights for policy 1, policy_version 9450 (0.0010) [2023-10-12 20:20:32,980][44959] Updated weights for policy 1, policy_version 9460 (0.0007) [2023-10-12 20:20:33,348][44959] Updated weights for policy 1, policy_version 9470 (0.0008) [2023-10-12 20:20:35,709][44958] Updated weights for policy 0, policy_version 9410 (0.0009) [2023-10-12 20:20:36,073][44958] Updated weights for policy 0, policy_version 9420 (0.0008) [2023-10-12 20:20:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 19333120. Throughput: 0: 1628.0, 1: 1651.4. Samples: 4842976. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 20:20:36,444][43579] Avg episode reward: [(0, '245.990'), (1, '232.910')] [2023-10-12 20:20:36,444][44958] Updated weights for policy 0, policy_version 9430 (0.0008) [2023-10-12 20:20:36,821][44958] Updated weights for policy 0, policy_version 9440 (0.0007) [2023-10-12 20:20:37,340][44959] Updated weights for policy 1, policy_version 9480 (0.0009) [2023-10-12 20:20:37,706][44959] Updated weights for policy 1, policy_version 9490 (0.0007) [2023-10-12 20:20:38,071][44959] Updated weights for policy 1, policy_version 9500 (0.0007) [2023-10-12 20:20:41,018][44958] Updated weights for policy 0, policy_version 9450 (0.0007) [2023-10-12 20:20:41,403][44958] Updated weights for policy 0, policy_version 9460 (0.0009) [2023-10-12 20:20:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19398656. Throughput: 0: 1640.6, 1: 1653.5. Samples: 4863458. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 20:20:41,444][43579] Avg episode reward: [(0, '248.800'), (1, '235.260')] [2023-10-12 20:20:41,787][44958] Updated weights for policy 0, policy_version 9470 (0.0007) [2023-10-12 20:20:42,335][44959] Updated weights for policy 1, policy_version 9510 (0.0007) [2023-10-12 20:20:42,706][44959] Updated weights for policy 1, policy_version 9520 (0.0008) [2023-10-12 20:20:43,069][44959] Updated weights for policy 1, policy_version 9530 (0.0007) [2023-10-12 20:20:45,915][44958] Updated weights for policy 0, policy_version 9480 (0.0011) [2023-10-12 20:20:46,285][44958] Updated weights for policy 0, policy_version 9490 (0.0008) [2023-10-12 20:20:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 19464192. Throughput: 0: 1639.2, 1: 1660.4. Samples: 4883254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:20:46,443][43579] Avg episode reward: [(0, '250.180'), (1, '232.200')] [2023-10-12 20:20:46,648][44958] Updated weights for policy 0, policy_version 9500 (0.0010) [2023-10-12 20:20:47,123][44959] Updated weights for policy 1, policy_version 9540 (0.0009) [2023-10-12 20:20:47,502][44959] Updated weights for policy 1, policy_version 9550 (0.0010) [2023-10-12 20:20:47,866][44959] Updated weights for policy 1, policy_version 9560 (0.0010) [2023-10-12 20:20:50,841][44958] Updated weights for policy 0, policy_version 9510 (0.0010) [2023-10-12 20:20:51,225][44958] Updated weights for policy 0, policy_version 9520 (0.0008) [2023-10-12 20:20:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19529728. Throughput: 0: 1638.1, 1: 1660.7. Samples: 4892620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:20:51,444][43579] Avg episode reward: [(0, '251.740'), (1, '233.780')] [2023-10-12 20:20:51,598][44958] Updated weights for policy 0, policy_version 9530 (0.0009) [2023-10-12 20:20:52,137][44959] Updated weights for policy 1, policy_version 9570 (0.0009) [2023-10-12 20:20:52,524][44959] Updated weights for policy 1, policy_version 9580 (0.0009) [2023-10-12 20:20:52,894][44959] Updated weights for policy 1, policy_version 9590 (0.0008) [2023-10-12 20:20:53,268][44959] Updated weights for policy 1, policy_version 9600 (0.0009) [2023-10-12 20:20:55,711][44958] Updated weights for policy 0, policy_version 9540 (0.0009) [2023-10-12 20:20:56,083][44958] Updated weights for policy 0, policy_version 9550 (0.0009) [2023-10-12 20:20:56,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19595264. Throughput: 0: 1635.5, 1: 1654.9. Samples: 4912624. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-10-12 20:20:56,444][43579] Avg episode reward: [(0, '252.780'), (1, '240.460')] [2023-10-12 20:20:56,447][44958] Updated weights for policy 0, policy_version 9560 (0.0009) [2023-10-12 20:20:57,310][44959] Updated weights for policy 1, policy_version 9610 (0.0011) [2023-10-12 20:20:57,678][44959] Updated weights for policy 1, policy_version 9620 (0.0009) [2023-10-12 20:20:58,058][44959] Updated weights for policy 1, policy_version 9630 (0.0011) [2023-10-12 20:21:00,623][44958] Updated weights for policy 0, policy_version 9570 (0.0009) [2023-10-12 20:21:00,988][44958] Updated weights for policy 0, policy_version 9580 (0.0008) [2023-10-12 20:21:01,357][44958] Updated weights for policy 0, policy_version 9590 (0.0008) [2023-10-12 20:21:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19660800. Throughput: 0: 1635.0, 1: 1656.5. Samples: 4932384. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-10-12 20:21:01,443][43579] Avg episode reward: [(0, '254.780'), (1, '248.110')] [2023-10-12 20:21:01,737][44958] Updated weights for policy 0, policy_version 9600 (0.0008) [2023-10-12 20:21:02,208][44959] Updated weights for policy 1, policy_version 9640 (0.0008) [2023-10-12 20:21:02,568][44959] Updated weights for policy 1, policy_version 9650 (0.0007) [2023-10-12 20:21:02,943][44959] Updated weights for policy 1, policy_version 9660 (0.0008) [2023-10-12 20:21:05,854][44958] Updated weights for policy 0, policy_version 9610 (0.0009) [2023-10-12 20:21:06,231][44958] Updated weights for policy 0, policy_version 9620 (0.0008) [2023-10-12 20:21:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19726336. Throughput: 0: 1633.2, 1: 1656.6. Samples: 4941890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:21:06,444][43579] Avg episode reward: [(0, '258.030'), (1, '245.500')] [2023-10-12 20:21:06,603][44958] Updated weights for policy 0, policy_version 9630 (0.0008) [2023-10-12 20:21:07,029][44959] Updated weights for policy 1, policy_version 9670 (0.0008) [2023-10-12 20:21:07,396][44959] Updated weights for policy 1, policy_version 9680 (0.0009) [2023-10-12 20:21:07,760][44959] Updated weights for policy 1, policy_version 9690 (0.0008) [2023-10-12 20:21:10,803][44958] Updated weights for policy 0, policy_version 9640 (0.0009) [2023-10-12 20:21:11,173][44958] Updated weights for policy 0, policy_version 9650 (0.0007) [2023-10-12 20:21:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 19791872. Throughput: 0: 1636.4, 1: 1656.7. Samples: 4962300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:21:11,443][43579] Avg episode reward: [(0, '259.130'), (1, '236.450')] [2023-10-12 20:21:11,546][44958] Updated weights for policy 0, policy_version 9660 (0.0008) [2023-10-12 20:21:11,969][44959] Updated weights for policy 1, policy_version 9700 (0.0007) [2023-10-12 20:21:12,348][44959] Updated weights for policy 1, policy_version 9710 (0.0010) [2023-10-12 20:21:12,707][44959] Updated weights for policy 1, policy_version 9720 (0.0007) [2023-10-12 20:21:15,651][44958] Updated weights for policy 0, policy_version 9670 (0.0009) [2023-10-12 20:21:16,023][44958] Updated weights for policy 0, policy_version 9680 (0.0008) [2023-10-12 20:21:16,391][44958] Updated weights for policy 0, policy_version 9690 (0.0009) [2023-10-12 20:21:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 19857408. Throughput: 0: 1636.5, 1: 1655.8. Samples: 4981852. Policy #0 lag: (min: 25.0, avg: 32.2, max: 57.0) [2023-10-12 20:21:16,443][43579] Avg episode reward: [(0, '258.290'), (1, '240.310')] [2023-10-12 20:21:16,698][44959] Updated weights for policy 1, policy_version 9730 (0.0008) [2023-10-12 20:21:17,069][44959] Updated weights for policy 1, policy_version 9740 (0.0007) [2023-10-12 20:21:17,446][44959] Updated weights for policy 1, policy_version 9750 (0.0007) [2023-10-12 20:21:17,817][44959] Updated weights for policy 1, policy_version 9760 (0.0008) [2023-10-12 20:21:20,737][44958] Updated weights for policy 0, policy_version 9700 (0.0008) [2023-10-12 20:21:21,112][44958] Updated weights for policy 0, policy_version 9710 (0.0008) [2023-10-12 20:21:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 19922944. Throughput: 0: 1640.5, 1: 1656.4. Samples: 4991336. Policy #0 lag: (min: 25.0, avg: 32.2, max: 57.0) [2023-10-12 20:21:21,443][43579] Avg episode reward: [(0, '261.880'), (1, '244.690')] [2023-10-12 20:21:21,487][44958] Updated weights for policy 0, policy_version 9720 (0.0007) [2023-10-12 20:21:21,946][44959] Updated weights for policy 1, policy_version 9770 (0.0008) [2023-10-12 20:21:22,317][44959] Updated weights for policy 1, policy_version 9780 (0.0008) [2023-10-12 20:21:22,682][44959] Updated weights for policy 1, policy_version 9790 (0.0009) [2023-10-12 20:21:25,682][44958] Updated weights for policy 0, policy_version 9730 (0.0007) [2023-10-12 20:21:26,057][44958] Updated weights for policy 0, policy_version 9740 (0.0007) [2023-10-12 20:21:26,429][44958] Updated weights for policy 0, policy_version 9750 (0.0009) [2023-10-12 20:21:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 19988480. Throughput: 0: 1639.6, 1: 1661.8. Samples: 5012022. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-12 20:21:26,443][43579] Avg episode reward: [(0, '263.940'), (1, '243.330')] [2023-10-12 20:21:26,656][44959] Updated weights for policy 1, policy_version 9800 (0.0007) [2023-10-12 20:21:26,802][44958] Updated weights for policy 0, policy_version 9760 (0.0009) [2023-10-12 20:21:27,024][44959] Updated weights for policy 1, policy_version 9810 (0.0008) [2023-10-12 20:21:27,395][44959] Updated weights for policy 1, policy_version 9820 (0.0010) [2023-10-12 20:21:30,773][44958] Updated weights for policy 0, policy_version 9770 (0.0007) [2023-10-12 20:21:31,147][44958] Updated weights for policy 0, policy_version 9780 (0.0008) [2023-10-12 20:21:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 20054016. Throughput: 0: 1639.2, 1: 1661.4. Samples: 5031778. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-12 20:21:31,443][43579] Avg episode reward: [(0, '263.210'), (1, '255.500')] [2023-10-12 20:21:31,523][44958] Updated weights for policy 0, policy_version 9790 (0.0008) [2023-10-12 20:21:31,629][44959] Updated weights for policy 1, policy_version 9830 (0.0009) [2023-10-12 20:21:31,994][44959] Updated weights for policy 1, policy_version 9840 (0.0008) [2023-10-12 20:21:32,366][44959] Updated weights for policy 1, policy_version 9850 (0.0007) [2023-10-12 20:21:35,522][44958] Updated weights for policy 0, policy_version 9800 (0.0008) [2023-10-12 20:21:35,897][44958] Updated weights for policy 0, policy_version 9810 (0.0008) [2023-10-12 20:21:36,268][44958] Updated weights for policy 0, policy_version 9820 (0.0009) [2023-10-12 20:21:36,407][44959] Updated weights for policy 1, policy_version 9860 (0.0008) [2023-10-12 20:21:36,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 20152320. Throughput: 0: 1646.2, 1: 1662.8. Samples: 5041522. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 20:21:36,444][43579] Avg episode reward: [(0, '260.630'), (1, '252.570')] [2023-10-12 20:21:36,803][44959] Updated weights for policy 1, policy_version 9870 (0.0010) [2023-10-12 20:21:37,169][44959] Updated weights for policy 1, policy_version 9880 (0.0009) [2023-10-12 20:21:40,654][44958] Updated weights for policy 0, policy_version 9830 (0.0009) [2023-10-12 20:21:41,035][44958] Updated weights for policy 0, policy_version 9840 (0.0008) [2023-10-12 20:21:41,294][44959] Updated weights for policy 1, policy_version 9890 (0.0009) [2023-10-12 20:21:41,407][44958] Updated weights for policy 0, policy_version 9850 (0.0008) [2023-10-12 20:21:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 20185088. Throughput: 0: 1650.0, 1: 1669.8. Samples: 5062014. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 20:21:41,443][43579] Avg episode reward: [(0, '258.830'), (1, '255.590')] [2023-10-12 20:21:41,662][44959] Updated weights for policy 1, policy_version 9900 (0.0008) [2023-10-12 20:21:42,023][44959] Updated weights for policy 1, policy_version 9910 (0.0009) [2023-10-12 20:21:42,402][44959] Updated weights for policy 1, policy_version 9920 (0.0009) [2023-10-12 20:21:45,466][44958] Updated weights for policy 0, policy_version 9860 (0.0008) [2023-10-12 20:21:45,833][44958] Updated weights for policy 0, policy_version 9870 (0.0009) [2023-10-12 20:21:46,217][44958] Updated weights for policy 0, policy_version 9880 (0.0007) [2023-10-12 20:21:46,443][43579] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 20250624. Throughput: 0: 1647.6, 1: 1671.9. Samples: 5081764. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-12 20:21:46,443][43579] Avg episode reward: [(0, '258.520'), (1, '258.040')] [2023-10-12 20:21:46,496][44959] Updated weights for policy 1, policy_version 9930 (0.0009) [2023-10-12 20:21:46,506][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000009888_10125312.pth... [2023-10-12 20:21:46,535][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000008352_8552448.pth [2023-10-12 20:21:46,867][44959] Updated weights for policy 1, policy_version 9940 (0.0008) [2023-10-12 20:21:47,235][44959] Updated weights for policy 1, policy_version 9950 (0.0010) [2023-10-12 20:21:47,302][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000009952_10190848.pth... [2023-10-12 20:21:47,332][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000008384_8585216.pth [2023-10-12 20:21:50,349][44958] Updated weights for policy 0, policy_version 9890 (0.0007) [2023-10-12 20:21:50,711][44958] Updated weights for policy 0, policy_version 9900 (0.0007) [2023-10-12 20:21:51,089][44958] Updated weights for policy 0, policy_version 9910 (0.0007) [2023-10-12 20:21:51,429][44959] Updated weights for policy 1, policy_version 9960 (0.0008) [2023-10-12 20:21:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 20316160. Throughput: 0: 1652.2, 1: 1671.4. Samples: 5091450. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-12 20:21:51,443][43579] Avg episode reward: [(0, '254.120'), (1, '256.430')] [2023-10-12 20:21:51,468][44958] Updated weights for policy 0, policy_version 9920 (0.0007) [2023-10-12 20:21:51,786][44959] Updated weights for policy 1, policy_version 9970 (0.0009) [2023-10-12 20:21:52,154][44959] Updated weights for policy 1, policy_version 9980 (0.0010) [2023-10-12 20:21:55,476][44958] Updated weights for policy 0, policy_version 9930 (0.0008) [2023-10-12 20:21:55,846][44958] Updated weights for policy 0, policy_version 9940 (0.0010) [2023-10-12 20:21:56,205][44958] Updated weights for policy 0, policy_version 9950 (0.0009) [2023-10-12 20:21:56,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13107.2). Total num frames: 20414464. Throughput: 0: 1651.1, 1: 1671.6. Samples: 5111824. Policy #0 lag: (min: 24.0, avg: 50.3, max: 56.0) [2023-10-12 20:21:56,443][43579] Avg episode reward: [(0, '256.330'), (1, '252.800')] [2023-10-12 20:21:56,474][44959] Updated weights for policy 1, policy_version 9990 (0.0010) [2023-10-12 20:21:56,851][44959] Updated weights for policy 1, policy_version 10000 (0.0008) [2023-10-12 20:21:57,215][44959] Updated weights for policy 1, policy_version 10010 (0.0008) [2023-10-12 20:22:00,363][44958] Updated weights for policy 0, policy_version 9960 (0.0007) [2023-10-12 20:22:00,744][44958] Updated weights for policy 0, policy_version 9970 (0.0010) [2023-10-12 20:22:01,119][44958] Updated weights for policy 0, policy_version 9980 (0.0008) [2023-10-12 20:22:01,167][44959] Updated weights for policy 1, policy_version 10020 (0.0007) [2023-10-12 20:22:01,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 20480000. Throughput: 0: 1648.2, 1: 1665.1. Samples: 5130948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:22:01,444][43579] Avg episode reward: [(0, '253.170'), (1, '246.810')] [2023-10-12 20:22:01,543][44959] Updated weights for policy 1, policy_version 10030 (0.0010) [2023-10-12 20:22:01,911][44959] Updated weights for policy 1, policy_version 10040 (0.0010) [2023-10-12 20:22:05,288][44958] Updated weights for policy 0, policy_version 9990 (0.0007) [2023-10-12 20:22:05,671][44958] Updated weights for policy 0, policy_version 10000 (0.0010) [2023-10-12 20:22:06,034][44958] Updated weights for policy 0, policy_version 10010 (0.0007) [2023-10-12 20:22:06,066][44959] Updated weights for policy 1, policy_version 10050 (0.0010) [2023-10-12 20:22:06,433][44959] Updated weights for policy 1, policy_version 10060 (0.0007) [2023-10-12 20:22:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 20545536. Throughput: 0: 1660.5, 1: 1664.2. Samples: 5140946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:22:06,444][43579] Avg episode reward: [(0, '257.250'), (1, '245.610')] [2023-10-12 20:22:06,811][44959] Updated weights for policy 1, policy_version 10070 (0.0008) [2023-10-12 20:22:07,185][44959] Updated weights for policy 1, policy_version 10080 (0.0008) [2023-10-12 20:22:10,227][44958] Updated weights for policy 0, policy_version 10020 (0.0008) [2023-10-12 20:22:10,600][44958] Updated weights for policy 0, policy_version 10030 (0.0007) [2023-10-12 20:22:10,965][44958] Updated weights for policy 0, policy_version 10040 (0.0008) [2023-10-12 20:22:11,280][44959] Updated weights for policy 1, policy_version 10090 (0.0007) [2023-10-12 20:22:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 20611072. Throughput: 0: 1655.5, 1: 1656.8. Samples: 5161074. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 20:22:11,443][43579] Avg episode reward: [(0, '258.450'), (1, '244.870')] [2023-10-12 20:22:11,664][44959] Updated weights for policy 1, policy_version 10100 (0.0010) [2023-10-12 20:22:12,036][44959] Updated weights for policy 1, policy_version 10110 (0.0009) [2023-10-12 20:22:15,146][44958] Updated weights for policy 0, policy_version 10050 (0.0007) [2023-10-12 20:22:15,524][44958] Updated weights for policy 0, policy_version 10060 (0.0008) [2023-10-12 20:22:15,889][44958] Updated weights for policy 0, policy_version 10070 (0.0008) [2023-10-12 20:22:16,211][44959] Updated weights for policy 1, policy_version 10120 (0.0009) [2023-10-12 20:22:16,263][44958] Updated weights for policy 0, policy_version 10080 (0.0008) [2023-10-12 20:22:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 20676608. Throughput: 0: 1645.8, 1: 1657.6. Samples: 5180432. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 20:22:16,443][43579] Avg episode reward: [(0, '259.670'), (1, '238.570')] [2023-10-12 20:22:16,582][44959] Updated weights for policy 1, policy_version 10130 (0.0008) [2023-10-12 20:22:16,941][44959] Updated weights for policy 1, policy_version 10140 (0.0008) [2023-10-12 20:22:20,518][44958] Updated weights for policy 0, policy_version 10090 (0.0010) [2023-10-12 20:22:20,902][44958] Updated weights for policy 0, policy_version 10100 (0.0008) [2023-10-12 20:22:20,982][44959] Updated weights for policy 1, policy_version 10150 (0.0009) [2023-10-12 20:22:21,269][44958] Updated weights for policy 0, policy_version 10110 (0.0008) [2023-10-12 20:22:21,353][44959] Updated weights for policy 1, policy_version 10160 (0.0007) [2023-10-12 20:22:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 20742144. Throughput: 0: 1651.0, 1: 1660.4. Samples: 5190532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:22:21,443][43579] Avg episode reward: [(0, '250.820'), (1, '246.220')] [2023-10-12 20:22:21,715][44959] Updated weights for policy 1, policy_version 10170 (0.0008) [2023-10-12 20:22:25,313][44958] Updated weights for policy 0, policy_version 10120 (0.0007) [2023-10-12 20:22:25,683][44958] Updated weights for policy 0, policy_version 10130 (0.0007) [2023-10-12 20:22:25,922][44959] Updated weights for policy 1, policy_version 10180 (0.0008) [2023-10-12 20:22:26,065][44958] Updated weights for policy 0, policy_version 10140 (0.0008) [2023-10-12 20:22:26,324][44959] Updated weights for policy 1, policy_version 10190 (0.0008) [2023-10-12 20:22:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 20807680. Throughput: 0: 1648.9, 1: 1659.0. Samples: 5210872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:22:26,443][43579] Avg episode reward: [(0, '254.450'), (1, '243.700')] [2023-10-12 20:22:26,685][44959] Updated weights for policy 1, policy_version 10200 (0.0008) [2023-10-12 20:22:30,266][44958] Updated weights for policy 0, policy_version 10150 (0.0010) [2023-10-12 20:22:30,632][44958] Updated weights for policy 0, policy_version 10160 (0.0008) [2023-10-12 20:22:30,742][44959] Updated weights for policy 1, policy_version 10210 (0.0008) [2023-10-12 20:22:31,016][44958] Updated weights for policy 0, policy_version 10170 (0.0009) [2023-10-12 20:22:31,112][44959] Updated weights for policy 1, policy_version 10220 (0.0009) [2023-10-12 20:22:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 20873216. Throughput: 0: 1645.0, 1: 1642.6. Samples: 5229706. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-12 20:22:31,443][43579] Avg episode reward: [(0, '244.920'), (1, '256.540')] [2023-10-12 20:22:31,487][44959] Updated weights for policy 1, policy_version 10230 (0.0008) [2023-10-12 20:22:31,853][44959] Updated weights for policy 1, policy_version 10240 (0.0009) [2023-10-12 20:22:35,085][44958] Updated weights for policy 0, policy_version 10180 (0.0009) [2023-10-12 20:22:35,464][44958] Updated weights for policy 0, policy_version 10190 (0.0009) [2023-10-12 20:22:35,831][44958] Updated weights for policy 0, policy_version 10200 (0.0007) [2023-10-12 20:22:36,086][44959] Updated weights for policy 1, policy_version 10250 (0.0009) [2023-10-12 20:22:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 20938752. Throughput: 0: 1652.4, 1: 1647.4. Samples: 5239938. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-12 20:22:36,443][43579] Avg episode reward: [(0, '245.570'), (1, '258.770')] [2023-10-12 20:22:36,459][44959] Updated weights for policy 1, policy_version 10260 (0.0008) [2023-10-12 20:22:36,833][44959] Updated weights for policy 1, policy_version 10270 (0.0008) [2023-10-12 20:22:39,995][44958] Updated weights for policy 0, policy_version 10210 (0.0007) [2023-10-12 20:22:40,369][44958] Updated weights for policy 0, policy_version 10220 (0.0009) [2023-10-12 20:22:40,735][44958] Updated weights for policy 0, policy_version 10230 (0.0008) [2023-10-12 20:22:40,920][44959] Updated weights for policy 1, policy_version 10280 (0.0007) [2023-10-12 20:22:41,112][44958] Updated weights for policy 0, policy_version 10240 (0.0007) [2023-10-12 20:22:41,293][44959] Updated weights for policy 1, policy_version 10290 (0.0009) [2023-10-12 20:22:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 21004288. Throughput: 0: 1648.0, 1: 1648.9. Samples: 5260186. Policy #0 lag: (min: 5.0, avg: 8.7, max: 37.0) [2023-10-12 20:22:41,443][43579] Avg episode reward: [(0, '246.030'), (1, '258.490')] [2023-10-12 20:22:41,669][44959] Updated weights for policy 1, policy_version 10300 (0.0009) [2023-10-12 20:22:45,354][44958] Updated weights for policy 0, policy_version 10250 (0.0008) [2023-10-12 20:22:45,734][44958] Updated weights for policy 0, policy_version 10260 (0.0010) [2023-10-12 20:22:45,848][44959] Updated weights for policy 1, policy_version 10310 (0.0009) [2023-10-12 20:22:46,091][44958] Updated weights for policy 0, policy_version 10270 (0.0009) [2023-10-12 20:22:46,214][44959] Updated weights for policy 1, policy_version 10320 (0.0008) [2023-10-12 20:22:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 21069824. Throughput: 0: 1645.1, 1: 1650.9. Samples: 5279266. Policy #0 lag: (min: 5.0, avg: 8.7, max: 37.0) [2023-10-12 20:22:46,443][43579] Avg episode reward: [(0, '246.750'), (1, '261.080')] [2023-10-12 20:22:46,580][44959] Updated weights for policy 1, policy_version 10330 (0.0007) [2023-10-12 20:22:46,796][44583] Saving new best policy, reward=261.080! [2023-10-12 20:22:50,253][44958] Updated weights for policy 0, policy_version 10280 (0.0008) [2023-10-12 20:22:50,626][44958] Updated weights for policy 0, policy_version 10290 (0.0007) [2023-10-12 20:22:50,718][44959] Updated weights for policy 1, policy_version 10340 (0.0008) [2023-10-12 20:22:51,003][44958] Updated weights for policy 0, policy_version 10300 (0.0008) [2023-10-12 20:22:51,084][44959] Updated weights for policy 1, policy_version 10350 (0.0009) [2023-10-12 20:22:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 21135360. Throughput: 0: 1640.9, 1: 1658.7. Samples: 5289428. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 20:22:51,443][43579] Avg episode reward: [(0, '255.100'), (1, '259.510')] [2023-10-12 20:22:51,465][44959] Updated weights for policy 1, policy_version 10360 (0.0009) [2023-10-12 20:22:55,269][44958] Updated weights for policy 0, policy_version 10310 (0.0008) [2023-10-12 20:22:55,579][44959] Updated weights for policy 1, policy_version 10370 (0.0008) [2023-10-12 20:22:55,634][44958] Updated weights for policy 0, policy_version 10320 (0.0007) [2023-10-12 20:22:55,948][44959] Updated weights for policy 1, policy_version 10380 (0.0008) [2023-10-12 20:22:56,006][44958] Updated weights for policy 0, policy_version 10330 (0.0007) [2023-10-12 20:22:56,311][44959] Updated weights for policy 1, policy_version 10390 (0.0009) [2023-10-12 20:22:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21200896. Throughput: 0: 1644.3, 1: 1660.8. Samples: 5309804. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 20:22:56,443][43579] Avg episode reward: [(0, '258.170'), (1, '258.390')] [2023-10-12 20:22:56,681][44959] Updated weights for policy 1, policy_version 10400 (0.0009) [2023-10-12 20:23:00,232][44958] Updated weights for policy 0, policy_version 10340 (0.0009) [2023-10-12 20:23:00,609][44958] Updated weights for policy 0, policy_version 10350 (0.0011) [2023-10-12 20:23:00,991][44958] Updated weights for policy 0, policy_version 10360 (0.0010) [2023-10-12 20:23:01,025][44959] Updated weights for policy 1, policy_version 10410 (0.0007) [2023-10-12 20:23:01,401][44959] Updated weights for policy 1, policy_version 10420 (0.0008) [2023-10-12 20:23:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 21266432. Throughput: 0: 1645.1, 1: 1647.0. Samples: 5328576. Policy #0 lag: (min: 22.0, avg: 23.2, max: 46.0) [2023-10-12 20:23:01,443][43579] Avg episode reward: [(0, '258.120'), (1, '242.440')] [2023-10-12 20:23:01,771][44959] Updated weights for policy 1, policy_version 10430 (0.0009) [2023-10-12 20:23:05,066][44958] Updated weights for policy 0, policy_version 10370 (0.0009) [2023-10-12 20:23:05,433][44958] Updated weights for policy 0, policy_version 10380 (0.0011) [2023-10-12 20:23:05,807][44958] Updated weights for policy 0, policy_version 10390 (0.0009) [2023-10-12 20:23:05,925][44959] Updated weights for policy 1, policy_version 10440 (0.0008) [2023-10-12 20:23:06,180][44958] Updated weights for policy 0, policy_version 10400 (0.0008) [2023-10-12 20:23:06,293][44959] Updated weights for policy 1, policy_version 10450 (0.0010) [2023-10-12 20:23:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 21331968. Throughput: 0: 1644.0, 1: 1652.3. Samples: 5338864. Policy #0 lag: (min: 22.0, avg: 23.2, max: 46.0) [2023-10-12 20:23:06,444][43579] Avg episode reward: [(0, '259.490'), (1, '235.780')] [2023-10-12 20:23:06,664][44959] Updated weights for policy 1, policy_version 10460 (0.0009) [2023-10-12 20:23:10,402][44958] Updated weights for policy 0, policy_version 10410 (0.0007) [2023-10-12 20:23:10,775][44958] Updated weights for policy 0, policy_version 10420 (0.0008) [2023-10-12 20:23:10,903][44959] Updated weights for policy 1, policy_version 10470 (0.0007) [2023-10-12 20:23:11,150][44958] Updated weights for policy 0, policy_version 10430 (0.0008) [2023-10-12 20:23:11,280][44959] Updated weights for policy 1, policy_version 10480 (0.0009) [2023-10-12 20:23:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 21397504. Throughput: 0: 1642.0, 1: 1651.0. Samples: 5359058. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 20:23:11,443][43579] Avg episode reward: [(0, '260.610'), (1, '231.570')] [2023-10-12 20:23:11,652][44959] Updated weights for policy 1, policy_version 10490 (0.0009) [2023-10-12 20:23:15,498][44958] Updated weights for policy 0, policy_version 10440 (0.0007) [2023-10-12 20:23:15,722][44959] Updated weights for policy 1, policy_version 10500 (0.0008) [2023-10-12 20:23:15,863][44958] Updated weights for policy 0, policy_version 10450 (0.0009) [2023-10-12 20:23:16,081][44959] Updated weights for policy 1, policy_version 10510 (0.0009) [2023-10-12 20:23:16,229][44958] Updated weights for policy 0, policy_version 10460 (0.0007) [2023-10-12 20:23:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 21463040. Throughput: 0: 1640.5, 1: 1649.5. Samples: 5377754. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 20:23:16,444][43579] Avg episode reward: [(0, '259.710'), (1, '231.710')] [2023-10-12 20:23:16,454][44959] Updated weights for policy 1, policy_version 10520 (0.0009) [2023-10-12 20:23:20,298][44958] Updated weights for policy 0, policy_version 10470 (0.0008) [2023-10-12 20:23:20,681][44958] Updated weights for policy 0, policy_version 10480 (0.0009) [2023-10-12 20:23:20,735][44959] Updated weights for policy 1, policy_version 10530 (0.0009) [2023-10-12 20:23:21,060][44958] Updated weights for policy 0, policy_version 10490 (0.0010) [2023-10-12 20:23:21,111][44959] Updated weights for policy 1, policy_version 10540 (0.0008) [2023-10-12 20:23:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 21528576. Throughput: 0: 1634.3, 1: 1655.3. Samples: 5387974. Policy #0 lag: (min: 28.0, avg: 34.5, max: 60.0) [2023-10-12 20:23:21,444][43579] Avg episode reward: [(0, '258.920'), (1, '230.680')] [2023-10-12 20:23:21,488][44959] Updated weights for policy 1, policy_version 10550 (0.0008) [2023-10-12 20:23:21,856][44959] Updated weights for policy 1, policy_version 10560 (0.0008) [2023-10-12 20:23:25,306][44958] Updated weights for policy 0, policy_version 10500 (0.0007) [2023-10-12 20:23:25,678][44958] Updated weights for policy 0, policy_version 10510 (0.0007) [2023-10-12 20:23:25,800][44959] Updated weights for policy 1, policy_version 10570 (0.0007) [2023-10-12 20:23:26,041][44958] Updated weights for policy 0, policy_version 10520 (0.0007) [2023-10-12 20:23:26,172][44959] Updated weights for policy 1, policy_version 10580 (0.0007) [2023-10-12 20:23:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 21594112. Throughput: 0: 1636.2, 1: 1656.7. Samples: 5408368. Policy #0 lag: (min: 28.0, avg: 34.5, max: 60.0) [2023-10-12 20:23:26,444][43579] Avg episode reward: [(0, '257.430'), (1, '243.420')] [2023-10-12 20:23:26,541][44959] Updated weights for policy 1, policy_version 10590 (0.0010) [2023-10-12 20:23:30,332][44958] Updated weights for policy 0, policy_version 10530 (0.0008) [2023-10-12 20:23:30,712][44959] Updated weights for policy 1, policy_version 10600 (0.0009) [2023-10-12 20:23:30,739][44958] Updated weights for policy 0, policy_version 10540 (0.0008) [2023-10-12 20:23:31,078][44959] Updated weights for policy 1, policy_version 10610 (0.0009) [2023-10-12 20:23:31,110][44958] Updated weights for policy 0, policy_version 10550 (0.0008) [2023-10-12 20:23:31,442][43579] Fps is (10 sec: 9830.6, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 21626880. Throughput: 0: 1631.7, 1: 1648.0. Samples: 5426852. Policy #0 lag: (min: 28.0, avg: 34.5, max: 60.0) [2023-10-12 20:23:31,443][43579] Avg episode reward: [(0, '257.660'), (1, '250.420')] [2023-10-12 20:23:31,447][44959] Updated weights for policy 1, policy_version 10620 (0.0009) [2023-10-12 20:23:31,481][44958] Updated weights for policy 0, policy_version 10560 (0.0009) [2023-10-12 20:23:35,404][44959] Updated weights for policy 1, policy_version 10630 (0.0008) [2023-10-12 20:23:35,541][44958] Updated weights for policy 0, policy_version 10570 (0.0008) [2023-10-12 20:23:35,781][44959] Updated weights for policy 1, policy_version 10640 (0.0007) [2023-10-12 20:23:35,910][44958] Updated weights for policy 0, policy_version 10580 (0.0007) [2023-10-12 20:23:36,138][44959] Updated weights for policy 1, policy_version 10650 (0.0008) [2023-10-12 20:23:36,279][44958] Updated weights for policy 0, policy_version 10590 (0.0008) [2023-10-12 20:23:36,443][43579] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 21757952. Throughput: 0: 1631.2, 1: 1655.5. Samples: 5437328. Policy #0 lag: (min: 25.0, avg: 41.9, max: 57.0) [2023-10-12 20:23:36,443][43579] Avg episode reward: [(0, '258.210'), (1, '261.930')] [2023-10-12 20:23:36,444][44583] Saving new best policy, reward=261.930! [2023-10-12 20:23:40,410][44959] Updated weights for policy 1, policy_version 10660 (0.0008) [2023-10-12 20:23:40,479][44958] Updated weights for policy 0, policy_version 10600 (0.0008) [2023-10-12 20:23:40,774][44959] Updated weights for policy 1, policy_version 10670 (0.0007) [2023-10-12 20:23:40,855][44958] Updated weights for policy 0, policy_version 10610 (0.0007) [2023-10-12 20:23:41,149][44959] Updated weights for policy 1, policy_version 10680 (0.0007) [2023-10-12 20:23:41,221][44958] Updated weights for policy 0, policy_version 10620 (0.0008) [2023-10-12 20:23:41,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 21790720. Throughput: 0: 1630.7, 1: 1649.6. Samples: 5457418. Policy #0 lag: (min: 30.0, avg: 37.5, max: 62.0) [2023-10-12 20:23:41,443][43579] Avg episode reward: [(0, '255.890'), (1, '262.080')] [2023-10-12 20:23:41,446][44583] Saving new best policy, reward=262.080! [2023-10-12 20:23:45,143][44959] Updated weights for policy 1, policy_version 10690 (0.0008) [2023-10-12 20:23:45,470][44958] Updated weights for policy 0, policy_version 10630 (0.0007) [2023-10-12 20:23:45,507][44959] Updated weights for policy 1, policy_version 10700 (0.0008) [2023-10-12 20:23:45,848][44958] Updated weights for policy 0, policy_version 10640 (0.0007) [2023-10-12 20:23:45,878][44959] Updated weights for policy 1, policy_version 10710 (0.0008) [2023-10-12 20:23:46,218][44958] Updated weights for policy 0, policy_version 10650 (0.0008) [2023-10-12 20:23:46,242][44959] Updated weights for policy 1, policy_version 10720 (0.0008) [2023-10-12 20:23:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 21889024. Throughput: 0: 1631.1, 1: 1639.1. Samples: 5475734. Policy #0 lag: (min: 30.0, avg: 37.5, max: 62.0) [2023-10-12 20:23:46,443][43579] Avg episode reward: [(0, '255.950'), (1, '266.750')] [2023-10-12 20:23:46,453][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000010656_10911744.pth... [2023-10-12 20:23:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000010720_10977280.pth... [2023-10-12 20:23:46,483][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000009120_9338880.pth [2023-10-12 20:23:46,494][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000009152_9371648.pth [2023-10-12 20:23:46,498][44583] Saving new best policy, reward=266.750! [2023-10-12 20:23:50,260][44958] Updated weights for policy 0, policy_version 10660 (0.0009) [2023-10-12 20:23:50,582][44959] Updated weights for policy 1, policy_version 10730 (0.0007) [2023-10-12 20:23:50,630][44958] Updated weights for policy 0, policy_version 10670 (0.0008) [2023-10-12 20:23:50,941][44959] Updated weights for policy 1, policy_version 10740 (0.0008) [2023-10-12 20:23:50,993][44958] Updated weights for policy 0, policy_version 10680 (0.0009) [2023-10-12 20:23:51,307][44959] Updated weights for policy 1, policy_version 10750 (0.0008) [2023-10-12 20:23:51,443][43579] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 21954560. Throughput: 0: 1629.0, 1: 1650.0. Samples: 5486420. Policy #0 lag: (min: 27.0, avg: 42.6, max: 59.0) [2023-10-12 20:23:51,444][43579] Avg episode reward: [(0, '255.820'), (1, '259.100')] [2023-10-12 20:23:55,317][44958] Updated weights for policy 0, policy_version 10690 (0.0007) [2023-10-12 20:23:55,521][44959] Updated weights for policy 1, policy_version 10760 (0.0009) [2023-10-12 20:23:55,685][44958] Updated weights for policy 0, policy_version 10700 (0.0008) [2023-10-12 20:23:55,894][44959] Updated weights for policy 1, policy_version 10770 (0.0009) [2023-10-12 20:23:56,054][44958] Updated weights for policy 0, policy_version 10710 (0.0008) [2023-10-12 20:23:56,259][44959] Updated weights for policy 1, policy_version 10780 (0.0008) [2023-10-12 20:23:56,428][44958] Updated weights for policy 0, policy_version 10720 (0.0009) [2023-10-12 20:23:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 22020096. Throughput: 0: 1627.8, 1: 1653.8. Samples: 5506730. Policy #0 lag: (min: 27.0, avg: 42.6, max: 59.0) [2023-10-12 20:23:56,444][43579] Avg episode reward: [(0, '261.540'), (1, '255.440')] [2023-10-12 20:24:00,503][44959] Updated weights for policy 1, policy_version 10790 (0.0008) [2023-10-12 20:24:00,772][44958] Updated weights for policy 0, policy_version 10730 (0.0007) [2023-10-12 20:24:00,871][44959] Updated weights for policy 1, policy_version 10800 (0.0009) [2023-10-12 20:24:01,150][44958] Updated weights for policy 0, policy_version 10740 (0.0008) [2023-10-12 20:24:01,244][44959] Updated weights for policy 1, policy_version 10810 (0.0008) [2023-10-12 20:24:01,443][43579] Fps is (10 sec: 6553.5, 60 sec: 12561.0, 300 sec: 12996.1). Total num frames: 22020096. Throughput: 0: 1632.1, 1: 1641.8. Samples: 5525078. Policy #0 lag: (min: 27.0, avg: 42.6, max: 59.0) [2023-10-12 20:24:01,444][43579] Avg episode reward: [(0, '257.220'), (1, '252.110')] [2023-10-12 20:24:01,517][44958] Updated weights for policy 0, policy_version 10750 (0.0008) [2023-10-12 20:24:05,333][44959] Updated weights for policy 1, policy_version 10820 (0.0008) [2023-10-12 20:24:05,513][44958] Updated weights for policy 0, policy_version 10760 (0.0007) [2023-10-12 20:24:05,704][44959] Updated weights for policy 1, policy_version 10830 (0.0007) [2023-10-12 20:24:05,889][44958] Updated weights for policy 0, policy_version 10770 (0.0008) [2023-10-12 20:24:06,074][44959] Updated weights for policy 1, policy_version 10840 (0.0009) [2023-10-12 20:24:06,261][44958] Updated weights for policy 0, policy_version 10780 (0.0010) [2023-10-12 20:24:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 22151168. Throughput: 0: 1633.1, 1: 1649.2. Samples: 5535680. Policy #0 lag: (min: 2.0, avg: 2.4, max: 12.0) [2023-10-12 20:24:06,444][43579] Avg episode reward: [(0, '261.910'), (1, '254.100')] [2023-10-12 20:24:10,193][44959] Updated weights for policy 1, policy_version 10850 (0.0010) [2023-10-12 20:24:10,560][44959] Updated weights for policy 1, policy_version 10860 (0.0007) [2023-10-12 20:24:10,596][44958] Updated weights for policy 0, policy_version 10790 (0.0010) [2023-10-12 20:24:10,936][44959] Updated weights for policy 1, policy_version 10870 (0.0009) [2023-10-12 20:24:10,966][44958] Updated weights for policy 0, policy_version 10800 (0.0010) [2023-10-12 20:24:11,304][44959] Updated weights for policy 1, policy_version 10880 (0.0008) [2023-10-12 20:24:11,352][44958] Updated weights for policy 0, policy_version 10810 (0.0008) [2023-10-12 20:24:11,442][43579] Fps is (10 sec: 16384.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22183936. Throughput: 0: 1626.9, 1: 1646.8. Samples: 5555686. Policy #0 lag: (min: 2.0, avg: 2.4, max: 12.0) [2023-10-12 20:24:11,443][43579] Avg episode reward: [(0, '259.910'), (1, '252.180')] [2023-10-12 20:24:15,503][44959] Updated weights for policy 1, policy_version 10890 (0.0007) [2023-10-12 20:24:15,513][44958] Updated weights for policy 0, policy_version 10820 (0.0009) [2023-10-12 20:24:15,868][44959] Updated weights for policy 1, policy_version 10900 (0.0008) [2023-10-12 20:24:15,885][44958] Updated weights for policy 0, policy_version 10830 (0.0009) [2023-10-12 20:24:16,233][44959] Updated weights for policy 1, policy_version 10910 (0.0008) [2023-10-12 20:24:16,249][44958] Updated weights for policy 0, policy_version 10840 (0.0008) [2023-10-12 20:24:16,443][43579] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 22249472. Throughput: 0: 1635.9, 1: 1638.1. Samples: 5574182. Policy #0 lag: (min: 29.0, avg: 36.9, max: 61.0) [2023-10-12 20:24:16,444][43579] Avg episode reward: [(0, '258.520'), (1, '256.730')] [2023-10-12 20:24:20,474][44959] Updated weights for policy 1, policy_version 10920 (0.0010) [2023-10-12 20:24:20,526][44958] Updated weights for policy 0, policy_version 10850 (0.0010) [2023-10-12 20:24:20,834][44959] Updated weights for policy 1, policy_version 10930 (0.0008) [2023-10-12 20:24:20,907][44958] Updated weights for policy 0, policy_version 10860 (0.0007) [2023-10-12 20:24:21,197][44959] Updated weights for policy 1, policy_version 10940 (0.0007) [2023-10-12 20:24:21,266][44958] Updated weights for policy 0, policy_version 10870 (0.0008) [2023-10-12 20:24:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 22315008. Throughput: 0: 1631.5, 1: 1646.9. Samples: 5584854. Policy #0 lag: (min: 29.0, avg: 36.9, max: 61.0) [2023-10-12 20:24:21,443][43579] Avg episode reward: [(0, '266.100'), (1, '253.000')] [2023-10-12 20:24:21,647][44958] Updated weights for policy 0, policy_version 10880 (0.0011) [2023-10-12 20:24:25,360][44959] Updated weights for policy 1, policy_version 10950 (0.0008) [2023-10-12 20:24:25,725][44959] Updated weights for policy 1, policy_version 10960 (0.0007) [2023-10-12 20:24:25,737][44958] Updated weights for policy 0, policy_version 10890 (0.0008) [2023-10-12 20:24:26,089][44959] Updated weights for policy 1, policy_version 10970 (0.0007) [2023-10-12 20:24:26,100][44958] Updated weights for policy 0, policy_version 10900 (0.0007) [2023-10-12 20:24:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22380544. Throughput: 0: 1628.2, 1: 1648.4. Samples: 5604866. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-12 20:24:26,443][43579] Avg episode reward: [(0, '266.370'), (1, '258.160')] [2023-10-12 20:24:26,472][44958] Updated weights for policy 0, policy_version 10910 (0.0008) [2023-10-12 20:24:30,157][44959] Updated weights for policy 1, policy_version 10980 (0.0009) [2023-10-12 20:24:30,521][44959] Updated weights for policy 1, policy_version 10990 (0.0009) [2023-10-12 20:24:30,626][44958] Updated weights for policy 0, policy_version 10920 (0.0008) [2023-10-12 20:24:30,887][44959] Updated weights for policy 1, policy_version 11000 (0.0009) [2023-10-12 20:24:30,991][44958] Updated weights for policy 0, policy_version 10930 (0.0008) [2023-10-12 20:24:31,360][44958] Updated weights for policy 0, policy_version 10940 (0.0007) [2023-10-12 20:24:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 22446080. Throughput: 0: 1633.2, 1: 1648.0. Samples: 5623388. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-12 20:24:31,443][43579] Avg episode reward: [(0, '266.110'), (1, '253.270')] [2023-10-12 20:24:34,995][44959] Updated weights for policy 1, policy_version 11010 (0.0007) [2023-10-12 20:24:35,359][44959] Updated weights for policy 1, policy_version 11020 (0.0007) [2023-10-12 20:24:35,529][44958] Updated weights for policy 0, policy_version 10950 (0.0007) [2023-10-12 20:24:35,725][44959] Updated weights for policy 1, policy_version 11030 (0.0008) [2023-10-12 20:24:35,902][44958] Updated weights for policy 0, policy_version 10960 (0.0009) [2023-10-12 20:24:36,087][44959] Updated weights for policy 1, policy_version 11040 (0.0009) [2023-10-12 20:24:36,278][44958] Updated weights for policy 0, policy_version 10970 (0.0008) [2023-10-12 20:24:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 22511616. Throughput: 0: 1631.3, 1: 1653.3. Samples: 5634228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:24:36,443][43579] Avg episode reward: [(0, '267.780'), (1, '254.630')] [2023-10-12 20:24:36,491][44518] Saving new best policy, reward=267.780! [2023-10-12 20:24:40,178][44959] Updated weights for policy 1, policy_version 11050 (0.0009) [2023-10-12 20:24:40,368][44958] Updated weights for policy 0, policy_version 10980 (0.0010) [2023-10-12 20:24:40,547][44959] Updated weights for policy 1, policy_version 11060 (0.0007) [2023-10-12 20:24:40,741][44958] Updated weights for policy 0, policy_version 10990 (0.0008) [2023-10-12 20:24:40,914][44959] Updated weights for policy 1, policy_version 11070 (0.0007) [2023-10-12 20:24:41,108][44958] Updated weights for policy 0, policy_version 11000 (0.0011) [2023-10-12 20:24:41,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 22609920. Throughput: 0: 1635.2, 1: 1644.3. Samples: 5654306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:24:41,444][43579] Avg episode reward: [(0, '270.390'), (1, '254.890')] [2023-10-12 20:24:41,445][44518] Saving new best policy, reward=270.390! [2023-10-12 20:24:45,097][44958] Updated weights for policy 0, policy_version 11010 (0.0009) [2023-10-12 20:24:45,348][44959] Updated weights for policy 1, policy_version 11080 (0.0007) [2023-10-12 20:24:45,474][44958] Updated weights for policy 0, policy_version 11020 (0.0008) [2023-10-12 20:24:45,721][44959] Updated weights for policy 1, policy_version 11090 (0.0007) [2023-10-12 20:24:45,846][44958] Updated weights for policy 0, policy_version 11030 (0.0007) [2023-10-12 20:24:46,091][44959] Updated weights for policy 1, policy_version 11100 (0.0009) [2023-10-12 20:24:46,212][44958] Updated weights for policy 0, policy_version 11040 (0.0008) [2023-10-12 20:24:46,443][43579] Fps is (10 sec: 16383.5, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 22675456. Throughput: 0: 1634.4, 1: 1648.5. Samples: 5672806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:24:46,444][43579] Avg episode reward: [(0, '272.170'), (1, '254.230')] [2023-10-12 20:24:46,453][44518] Saving new best policy, reward=272.170! [2023-10-12 20:24:49,952][44959] Updated weights for policy 1, policy_version 11110 (0.0008) [2023-10-12 20:24:50,319][44959] Updated weights for policy 1, policy_version 11120 (0.0008) [2023-10-12 20:24:50,570][44958] Updated weights for policy 0, policy_version 11050 (0.0008) [2023-10-12 20:24:50,692][44959] Updated weights for policy 1, policy_version 11130 (0.0008) [2023-10-12 20:24:50,950][44958] Updated weights for policy 0, policy_version 11060 (0.0009) [2023-10-12 20:24:51,313][44958] Updated weights for policy 0, policy_version 11070 (0.0009) [2023-10-12 20:24:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22740992. Throughput: 0: 1632.5, 1: 1657.1. Samples: 5683712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:24:51,444][43579] Avg episode reward: [(0, '269.940'), (1, '260.060')] [2023-10-12 20:24:55,010][44959] Updated weights for policy 1, policy_version 11140 (0.0008) [2023-10-12 20:24:55,381][44959] Updated weights for policy 1, policy_version 11150 (0.0009) [2023-10-12 20:24:55,643][44958] Updated weights for policy 0, policy_version 11080 (0.0007) [2023-10-12 20:24:55,747][44959] Updated weights for policy 1, policy_version 11160 (0.0007) [2023-10-12 20:24:56,027][44958] Updated weights for policy 0, policy_version 11090 (0.0007) [2023-10-12 20:24:56,391][44958] Updated weights for policy 0, policy_version 11100 (0.0010) [2023-10-12 20:24:56,442][43579] Fps is (10 sec: 9830.7, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 22773760. Throughput: 0: 1637.2, 1: 1650.1. Samples: 5703616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:24:56,443][43579] Avg episode reward: [(0, '268.720'), (1, '252.140')] [2023-10-12 20:24:59,705][44959] Updated weights for policy 1, policy_version 11170 (0.0009) [2023-10-12 20:25:00,070][44959] Updated weights for policy 1, policy_version 11180 (0.0009) [2023-10-12 20:25:00,441][44959] Updated weights for policy 1, policy_version 11190 (0.0008) [2023-10-12 20:25:00,780][44958] Updated weights for policy 0, policy_version 11110 (0.0008) [2023-10-12 20:25:00,806][44959] Updated weights for policy 1, policy_version 11200 (0.0009) [2023-10-12 20:25:01,157][44958] Updated weights for policy 0, policy_version 11120 (0.0007) [2023-10-12 20:25:01,442][43579] Fps is (10 sec: 9830.6, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 22839296. Throughput: 0: 1637.5, 1: 1646.0. Samples: 5721936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:25:01,443][43579] Avg episode reward: [(0, '271.420'), (1, '249.060')] [2023-10-12 20:25:01,535][44958] Updated weights for policy 0, policy_version 11130 (0.0009) [2023-10-12 20:25:04,902][44959] Updated weights for policy 1, policy_version 11210 (0.0007) [2023-10-12 20:25:05,265][44959] Updated weights for policy 1, policy_version 11220 (0.0010) [2023-10-12 20:25:05,599][44958] Updated weights for policy 0, policy_version 11140 (0.0007) [2023-10-12 20:25:05,627][44959] Updated weights for policy 1, policy_version 11230 (0.0009) [2023-10-12 20:25:05,969][44958] Updated weights for policy 0, policy_version 11150 (0.0008) [2023-10-12 20:25:06,346][44958] Updated weights for policy 0, policy_version 11160 (0.0009) [2023-10-12 20:25:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 22904832. Throughput: 0: 1632.3, 1: 1652.8. Samples: 5732684. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-12 20:25:06,443][43579] Avg episode reward: [(0, '270.270'), (1, '244.100')] [2023-10-12 20:25:09,965][44959] Updated weights for policy 1, policy_version 11240 (0.0010) [2023-10-12 20:25:10,338][44959] Updated weights for policy 1, policy_version 11250 (0.0009) [2023-10-12 20:25:10,634][44958] Updated weights for policy 0, policy_version 11170 (0.0008) [2023-10-12 20:25:10,706][44959] Updated weights for policy 1, policy_version 11260 (0.0008) [2023-10-12 20:25:11,019][44958] Updated weights for policy 0, policy_version 11180 (0.0009) [2023-10-12 20:25:11,396][44958] Updated weights for policy 0, policy_version 11190 (0.0009) [2023-10-12 20:25:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 22970368. Throughput: 0: 1633.4, 1: 1644.9. Samples: 5752390. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-12 20:25:11,444][43579] Avg episode reward: [(0, '262.840'), (1, '244.070')] [2023-10-12 20:25:11,763][44958] Updated weights for policy 0, policy_version 11200 (0.0009) [2023-10-12 20:25:14,625][44959] Updated weights for policy 1, policy_version 11270 (0.0009) [2023-10-12 20:25:14,993][44959] Updated weights for policy 1, policy_version 11280 (0.0011) [2023-10-12 20:25:15,363][44959] Updated weights for policy 1, policy_version 11290 (0.0011) [2023-10-12 20:25:16,022][44958] Updated weights for policy 0, policy_version 11210 (0.0009) [2023-10-12 20:25:16,382][44958] Updated weights for policy 0, policy_version 11220 (0.0010) [2023-10-12 20:25:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23035904. Throughput: 0: 1640.2, 1: 1648.0. Samples: 5771356. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-12 20:25:16,444][43579] Avg episode reward: [(0, '262.520'), (1, '240.860')] [2023-10-12 20:25:16,757][44958] Updated weights for policy 0, policy_version 11230 (0.0008) [2023-10-12 20:25:19,564][44959] Updated weights for policy 1, policy_version 11300 (0.0008) [2023-10-12 20:25:19,929][44959] Updated weights for policy 1, policy_version 11310 (0.0009) [2023-10-12 20:25:20,297][44959] Updated weights for policy 1, policy_version 11320 (0.0008) [2023-10-12 20:25:21,039][44958] Updated weights for policy 0, policy_version 11240 (0.0008) [2023-10-12 20:25:21,415][44958] Updated weights for policy 0, policy_version 11250 (0.0009) [2023-10-12 20:25:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23101440. Throughput: 0: 1627.6, 1: 1655.1. Samples: 5781950. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) [2023-10-12 20:25:21,443][43579] Avg episode reward: [(0, '263.770'), (1, '247.120')] [2023-10-12 20:25:21,782][44958] Updated weights for policy 0, policy_version 11260 (0.0010) [2023-10-12 20:25:24,590][44959] Updated weights for policy 1, policy_version 11330 (0.0008) [2023-10-12 20:25:24,967][44959] Updated weights for policy 1, policy_version 11340 (0.0011) [2023-10-12 20:25:25,330][44959] Updated weights for policy 1, policy_version 11350 (0.0009) [2023-10-12 20:25:25,688][44959] Updated weights for policy 1, policy_version 11360 (0.0007) [2023-10-12 20:25:26,044][44958] Updated weights for policy 0, policy_version 11270 (0.0008) [2023-10-12 20:25:26,412][44958] Updated weights for policy 0, policy_version 11280 (0.0010) [2023-10-12 20:25:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23166976. Throughput: 0: 1624.8, 1: 1646.6. Samples: 5801518. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-12 20:25:26,443][43579] Avg episode reward: [(0, '258.330'), (1, '246.570')] [2023-10-12 20:25:26,782][44958] Updated weights for policy 0, policy_version 11290 (0.0007) [2023-10-12 20:25:30,001][44959] Updated weights for policy 1, policy_version 11370 (0.0009) [2023-10-12 20:25:30,375][44959] Updated weights for policy 1, policy_version 11380 (0.0007) [2023-10-12 20:25:30,743][44959] Updated weights for policy 1, policy_version 11390 (0.0008) [2023-10-12 20:25:30,867][44958] Updated weights for policy 0, policy_version 11300 (0.0011) [2023-10-12 20:25:31,236][44958] Updated weights for policy 0, policy_version 11310 (0.0010) [2023-10-12 20:25:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23232512. Throughput: 0: 1642.0, 1: 1648.8. Samples: 5820888. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-12 20:25:31,443][43579] Avg episode reward: [(0, '255.590'), (1, '249.900')] [2023-10-12 20:25:31,608][44958] Updated weights for policy 0, policy_version 11320 (0.0010) [2023-10-12 20:25:34,816][44959] Updated weights for policy 1, policy_version 11400 (0.0007) [2023-10-12 20:25:35,189][44959] Updated weights for policy 1, policy_version 11410 (0.0008) [2023-10-12 20:25:35,503][44958] Updated weights for policy 0, policy_version 11330 (0.0007) [2023-10-12 20:25:35,549][44959] Updated weights for policy 1, policy_version 11420 (0.0007) [2023-10-12 20:25:35,889][44958] Updated weights for policy 0, policy_version 11340 (0.0008) [2023-10-12 20:25:36,252][44958] Updated weights for policy 0, policy_version 11350 (0.0008) [2023-10-12 20:25:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23298048. Throughput: 0: 1629.7, 1: 1656.9. Samples: 5831608. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:25:36,443][43579] Avg episode reward: [(0, '260.140'), (1, '250.440')] [2023-10-12 20:25:36,624][44958] Updated weights for policy 0, policy_version 11360 (0.0008) [2023-10-12 20:25:39,678][44959] Updated weights for policy 1, policy_version 11430 (0.0009) [2023-10-12 20:25:40,055][44959] Updated weights for policy 1, policy_version 11440 (0.0010) [2023-10-12 20:25:40,431][44959] Updated weights for policy 1, policy_version 11450 (0.0009) [2023-10-12 20:25:40,612][44958] Updated weights for policy 0, policy_version 11370 (0.0007) [2023-10-12 20:25:40,983][44958] Updated weights for policy 0, policy_version 11380 (0.0007) [2023-10-12 20:25:41,362][44958] Updated weights for policy 0, policy_version 11390 (0.0007) [2023-10-12 20:25:41,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23396352. Throughput: 0: 1634.2, 1: 1647.6. Samples: 5851298. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:25:41,443][43579] Avg episode reward: [(0, '263.590'), (1, '253.050')] [2023-10-12 20:25:44,434][44959] Updated weights for policy 1, policy_version 11460 (0.0007) [2023-10-12 20:25:44,797][44959] Updated weights for policy 1, policy_version 11470 (0.0009) [2023-10-12 20:25:45,166][44959] Updated weights for policy 1, policy_version 11480 (0.0008) [2023-10-12 20:25:45,626][44958] Updated weights for policy 0, policy_version 11400 (0.0007) [2023-10-12 20:25:46,000][44958] Updated weights for policy 0, policy_version 11410 (0.0009) [2023-10-12 20:25:46,382][44958] Updated weights for policy 0, policy_version 11420 (0.0010) [2023-10-12 20:25:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 23429120. Throughput: 0: 1629.8, 1: 1660.6. Samples: 5870004. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:25:46,443][43579] Avg episode reward: [(0, '257.510'), (1, '252.020')] [2023-10-12 20:25:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000011488_11763712.pth... [2023-10-12 20:25:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000009952_10190848.pth [2023-10-12 20:25:46,526][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000011424_11698176.pth... [2023-10-12 20:25:46,566][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000009888_10125312.pth [2023-10-12 20:25:49,236][44959] Updated weights for policy 1, policy_version 11490 (0.0007) [2023-10-12 20:25:49,604][44959] Updated weights for policy 1, policy_version 11500 (0.0007) [2023-10-12 20:25:49,982][44959] Updated weights for policy 1, policy_version 11510 (0.0007) [2023-10-12 20:25:50,347][44959] Updated weights for policy 1, policy_version 11520 (0.0007) [2023-10-12 20:25:50,775][44958] Updated weights for policy 0, policy_version 11430 (0.0008) [2023-10-12 20:25:51,154][44958] Updated weights for policy 0, policy_version 11440 (0.0008) [2023-10-12 20:25:51,442][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 23494656. Throughput: 0: 1632.0, 1: 1656.1. Samples: 5880646. Policy #0 lag: (min: 24.0, avg: 42.8, max: 56.0) [2023-10-12 20:25:51,443][43579] Avg episode reward: [(0, '258.120'), (1, '253.550')] [2023-10-12 20:25:51,523][44958] Updated weights for policy 0, policy_version 11450 (0.0008) [2023-10-12 20:25:54,500][44959] Updated weights for policy 1, policy_version 11530 (0.0008) [2023-10-12 20:25:54,859][44959] Updated weights for policy 1, policy_version 11540 (0.0008) [2023-10-12 20:25:55,231][44959] Updated weights for policy 1, policy_version 11550 (0.0007) [2023-10-12 20:25:55,886][44958] Updated weights for policy 0, policy_version 11460 (0.0008) [2023-10-12 20:25:56,270][44958] Updated weights for policy 0, policy_version 11470 (0.0009) [2023-10-12 20:25:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23560192. Throughput: 0: 1635.8, 1: 1643.6. Samples: 5899962. Policy #0 lag: (min: 24.0, avg: 42.8, max: 56.0) [2023-10-12 20:25:56,444][43579] Avg episode reward: [(0, '263.430'), (1, '242.550')] [2023-10-12 20:25:56,644][44958] Updated weights for policy 0, policy_version 11480 (0.0009) [2023-10-12 20:25:59,477][44959] Updated weights for policy 1, policy_version 11560 (0.0009) [2023-10-12 20:25:59,844][44959] Updated weights for policy 1, policy_version 11570 (0.0010) [2023-10-12 20:26:00,211][44959] Updated weights for policy 1, policy_version 11580 (0.0008) [2023-10-12 20:26:00,693][44958] Updated weights for policy 0, policy_version 11490 (0.0007) [2023-10-12 20:26:01,057][44958] Updated weights for policy 0, policy_version 11500 (0.0008) [2023-10-12 20:26:01,435][44958] Updated weights for policy 0, policy_version 11510 (0.0010) [2023-10-12 20:26:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23625728. Throughput: 0: 1636.0, 1: 1653.6. Samples: 5919386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:26:01,443][43579] Avg episode reward: [(0, '261.330'), (1, '237.330')] [2023-10-12 20:26:01,805][44958] Updated weights for policy 0, policy_version 11520 (0.0010) [2023-10-12 20:26:04,435][44959] Updated weights for policy 1, policy_version 11590 (0.0007) [2023-10-12 20:26:04,811][44959] Updated weights for policy 1, policy_version 11600 (0.0008) [2023-10-12 20:26:05,175][44959] Updated weights for policy 1, policy_version 11610 (0.0007) [2023-10-12 20:26:06,047][44958] Updated weights for policy 0, policy_version 11530 (0.0008) [2023-10-12 20:26:06,415][44958] Updated weights for policy 0, policy_version 11540 (0.0010) [2023-10-12 20:26:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23691264. Throughput: 0: 1638.0, 1: 1646.2. Samples: 5929742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:26:06,443][43579] Avg episode reward: [(0, '262.440'), (1, '225.020')] [2023-10-12 20:26:06,787][44958] Updated weights for policy 0, policy_version 11550 (0.0008) [2023-10-12 20:26:09,330][44959] Updated weights for policy 1, policy_version 11620 (0.0010) [2023-10-12 20:26:09,704][44959] Updated weights for policy 1, policy_version 11630 (0.0011) [2023-10-12 20:26:10,067][44959] Updated weights for policy 1, policy_version 11640 (0.0008) [2023-10-12 20:26:11,004][44958] Updated weights for policy 0, policy_version 11560 (0.0008) [2023-10-12 20:26:11,372][44958] Updated weights for policy 0, policy_version 11570 (0.0008) [2023-10-12 20:26:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23756800. Throughput: 0: 1643.6, 1: 1638.0. Samples: 5949186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:26:11,443][43579] Avg episode reward: [(0, '265.540'), (1, '230.980')] [2023-10-12 20:26:11,750][44958] Updated weights for policy 0, policy_version 11580 (0.0008) [2023-10-12 20:26:14,279][44959] Updated weights for policy 1, policy_version 11650 (0.0009) [2023-10-12 20:26:14,677][44959] Updated weights for policy 1, policy_version 11660 (0.0009) [2023-10-12 20:26:15,048][44959] Updated weights for policy 1, policy_version 11670 (0.0011) [2023-10-12 20:26:15,419][44959] Updated weights for policy 1, policy_version 11680 (0.0008) [2023-10-12 20:26:15,711][44958] Updated weights for policy 0, policy_version 11590 (0.0008) [2023-10-12 20:26:16,082][44958] Updated weights for policy 0, policy_version 11600 (0.0009) [2023-10-12 20:26:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23822336. Throughput: 0: 1629.3, 1: 1649.5. Samples: 5968434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:26:16,443][43579] Avg episode reward: [(0, '260.790'), (1, '227.320')] [2023-10-12 20:26:16,459][44958] Updated weights for policy 0, policy_version 11610 (0.0008) [2023-10-12 20:26:19,589][44959] Updated weights for policy 1, policy_version 11690 (0.0007) [2023-10-12 20:26:19,946][44959] Updated weights for policy 1, policy_version 11700 (0.0010) [2023-10-12 20:26:20,314][44959] Updated weights for policy 1, policy_version 11710 (0.0007) [2023-10-12 20:26:20,708][44958] Updated weights for policy 0, policy_version 11620 (0.0007) [2023-10-12 20:26:21,091][44958] Updated weights for policy 0, policy_version 11630 (0.0008) [2023-10-12 20:26:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23887872. Throughput: 0: 1632.2, 1: 1642.8. Samples: 5978980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:26:21,443][43579] Avg episode reward: [(0, '264.990'), (1, '237.810')] [2023-10-12 20:26:21,465][44958] Updated weights for policy 0, policy_version 11640 (0.0010) [2023-10-12 20:26:24,456][44959] Updated weights for policy 1, policy_version 11720 (0.0007) [2023-10-12 20:26:24,828][44959] Updated weights for policy 1, policy_version 11730 (0.0007) [2023-10-12 20:26:25,196][44959] Updated weights for policy 1, policy_version 11740 (0.0007) [2023-10-12 20:26:25,589][44958] Updated weights for policy 0, policy_version 11650 (0.0010) [2023-10-12 20:26:25,959][44958] Updated weights for policy 0, policy_version 11660 (0.0010) [2023-10-12 20:26:26,330][44958] Updated weights for policy 0, policy_version 11670 (0.0007) [2023-10-12 20:26:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 23953408. Throughput: 0: 1629.4, 1: 1638.1. Samples: 5998338. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:26:26,443][43579] Avg episode reward: [(0, '262.620'), (1, '240.480')] [2023-10-12 20:26:26,701][44958] Updated weights for policy 0, policy_version 11680 (0.0008) [2023-10-12 20:26:29,330][44959] Updated weights for policy 1, policy_version 11750 (0.0008) [2023-10-12 20:26:29,692][44959] Updated weights for policy 1, policy_version 11760 (0.0011) [2023-10-12 20:26:30,056][44959] Updated weights for policy 1, policy_version 11770 (0.0009) [2023-10-12 20:26:30,981][44958] Updated weights for policy 0, policy_version 11690 (0.0008) [2023-10-12 20:26:31,366][44958] Updated weights for policy 0, policy_version 11700 (0.0007) [2023-10-12 20:26:31,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 24018944. Throughput: 0: 1637.0, 1: 1643.3. Samples: 6017620. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:26:31,444][43579] Avg episode reward: [(0, '264.260'), (1, '256.270')] [2023-10-12 20:26:31,734][44958] Updated weights for policy 0, policy_version 11710 (0.0007) [2023-10-12 20:26:34,388][44959] Updated weights for policy 1, policy_version 11780 (0.0011) [2023-10-12 20:26:34,756][44959] Updated weights for policy 1, policy_version 11790 (0.0008) [2023-10-12 20:26:35,129][44959] Updated weights for policy 1, policy_version 11800 (0.0007) [2023-10-12 20:26:35,825][44958] Updated weights for policy 0, policy_version 11720 (0.0008) [2023-10-12 20:26:36,205][44958] Updated weights for policy 0, policy_version 11730 (0.0007) [2023-10-12 20:26:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 24084480. Throughput: 0: 1640.4, 1: 1640.3. Samples: 6028278. Policy #0 lag: (min: 15.0, avg: 20.0, max: 47.0) [2023-10-12 20:26:36,443][43579] Avg episode reward: [(0, '260.080'), (1, '251.740')] [2023-10-12 20:26:36,580][44958] Updated weights for policy 0, policy_version 11740 (0.0008) [2023-10-12 20:26:39,275][44959] Updated weights for policy 1, policy_version 11810 (0.0009) [2023-10-12 20:26:39,645][44959] Updated weights for policy 1, policy_version 11820 (0.0009) [2023-10-12 20:26:40,016][44959] Updated weights for policy 1, policy_version 11830 (0.0007) [2023-10-12 20:26:40,386][44959] Updated weights for policy 1, policy_version 11840 (0.0008) [2023-10-12 20:26:40,708][44958] Updated weights for policy 0, policy_version 11750 (0.0009) [2023-10-12 20:26:41,081][44958] Updated weights for policy 0, policy_version 11760 (0.0007) [2023-10-12 20:26:41,442][43579] Fps is (10 sec: 13107.6, 60 sec: 12561.1, 300 sec: 13218.3). Total num frames: 24150016. Throughput: 0: 1636.5, 1: 1648.9. Samples: 6047804. Policy #0 lag: (min: 15.0, avg: 20.0, max: 47.0) [2023-10-12 20:26:41,443][43579] Avg episode reward: [(0, '262.480'), (1, '258.250')] [2023-10-12 20:26:41,462][44958] Updated weights for policy 0, policy_version 11770 (0.0007) [2023-10-12 20:26:44,571][44959] Updated weights for policy 1, policy_version 11850 (0.0010) [2023-10-12 20:26:44,946][44959] Updated weights for policy 1, policy_version 11860 (0.0007) [2023-10-12 20:26:45,310][44959] Updated weights for policy 1, policy_version 11870 (0.0008) [2023-10-12 20:26:45,744][44958] Updated weights for policy 0, policy_version 11780 (0.0008) [2023-10-12 20:26:46,116][44958] Updated weights for policy 0, policy_version 11790 (0.0008) [2023-10-12 20:26:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 24215552. Throughput: 0: 1630.4, 1: 1645.8. Samples: 6066816. Policy #0 lag: (min: 9.0, avg: 16.1, max: 41.0) [2023-10-12 20:26:46,444][43579] Avg episode reward: [(0, '262.300'), (1, '257.140')] [2023-10-12 20:26:46,478][44958] Updated weights for policy 0, policy_version 11800 (0.0008) [2023-10-12 20:26:49,501][44959] Updated weights for policy 1, policy_version 11880 (0.0009) [2023-10-12 20:26:49,861][44959] Updated weights for policy 1, policy_version 11890 (0.0010) [2023-10-12 20:26:50,234][44959] Updated weights for policy 1, policy_version 11900 (0.0008) [2023-10-12 20:26:50,648][44958] Updated weights for policy 0, policy_version 11810 (0.0007) [2023-10-12 20:26:51,030][44958] Updated weights for policy 0, policy_version 11820 (0.0010) [2023-10-12 20:26:51,393][44958] Updated weights for policy 0, policy_version 11830 (0.0010) [2023-10-12 20:26:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24281088. Throughput: 0: 1632.9, 1: 1650.6. Samples: 6077498. Policy #0 lag: (min: 9.0, avg: 16.1, max: 41.0) [2023-10-12 20:26:51,443][43579] Avg episode reward: [(0, '260.950'), (1, '258.810')] [2023-10-12 20:26:51,769][44958] Updated weights for policy 0, policy_version 11840 (0.0011) [2023-10-12 20:26:54,408][44959] Updated weights for policy 1, policy_version 11910 (0.0008) [2023-10-12 20:26:54,774][44959] Updated weights for policy 1, policy_version 11920 (0.0010) [2023-10-12 20:26:55,143][44959] Updated weights for policy 1, policy_version 11930 (0.0007) [2023-10-12 20:26:55,841][44958] Updated weights for policy 0, policy_version 11850 (0.0010) [2023-10-12 20:26:56,221][44958] Updated weights for policy 0, policy_version 11860 (0.0009) [2023-10-12 20:26:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24346624. Throughput: 0: 1632.6, 1: 1652.2. Samples: 6097002. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-12 20:26:56,443][43579] Avg episode reward: [(0, '258.710'), (1, '251.980')] [2023-10-12 20:26:56,592][44958] Updated weights for policy 0, policy_version 11870 (0.0008) [2023-10-12 20:26:59,266][44959] Updated weights for policy 1, policy_version 11940 (0.0008) [2023-10-12 20:26:59,665][44959] Updated weights for policy 1, policy_version 11950 (0.0010) [2023-10-12 20:27:00,037][44959] Updated weights for policy 1, policy_version 11960 (0.0008) [2023-10-12 20:27:00,845][44958] Updated weights for policy 0, policy_version 11880 (0.0007) [2023-10-12 20:27:01,214][44958] Updated weights for policy 0, policy_version 11890 (0.0007) [2023-10-12 20:27:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24412160. Throughput: 0: 1637.3, 1: 1644.1. Samples: 6116096. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-12 20:27:01,444][43579] Avg episode reward: [(0, '262.800'), (1, '254.170')] [2023-10-12 20:27:01,593][44958] Updated weights for policy 0, policy_version 11900 (0.0010) [2023-10-12 20:27:04,196][44959] Updated weights for policy 1, policy_version 11970 (0.0009) [2023-10-12 20:27:04,564][44959] Updated weights for policy 1, policy_version 11980 (0.0008) [2023-10-12 20:27:04,920][44959] Updated weights for policy 1, policy_version 11990 (0.0009) [2023-10-12 20:27:05,296][44959] Updated weights for policy 1, policy_version 12000 (0.0009) [2023-10-12 20:27:05,775][44958] Updated weights for policy 0, policy_version 11910 (0.0009) [2023-10-12 20:27:06,161][44958] Updated weights for policy 0, policy_version 11920 (0.0010) [2023-10-12 20:27:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24477696. Throughput: 0: 1640.0, 1: 1644.0. Samples: 6126758. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-12 20:27:06,443][43579] Avg episode reward: [(0, '265.580'), (1, '247.760')] [2023-10-12 20:27:06,523][44958] Updated weights for policy 0, policy_version 11930 (0.0007) [2023-10-12 20:27:09,411][44959] Updated weights for policy 1, policy_version 12010 (0.0007) [2023-10-12 20:27:09,771][44959] Updated weights for policy 1, policy_version 12020 (0.0008) [2023-10-12 20:27:10,138][44959] Updated weights for policy 1, policy_version 12030 (0.0008) [2023-10-12 20:27:10,689][44958] Updated weights for policy 0, policy_version 11940 (0.0010) [2023-10-12 20:27:11,061][44958] Updated weights for policy 0, policy_version 11950 (0.0012) [2023-10-12 20:27:11,431][44958] Updated weights for policy 0, policy_version 11960 (0.0009) [2023-10-12 20:27:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24543232. Throughput: 0: 1640.6, 1: 1646.1. Samples: 6146240. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-12 20:27:11,443][43579] Avg episode reward: [(0, '262.830'), (1, '250.840')] [2023-10-12 20:27:14,344][44959] Updated weights for policy 1, policy_version 12040 (0.0010) [2023-10-12 20:27:14,717][44959] Updated weights for policy 1, policy_version 12050 (0.0007) [2023-10-12 20:27:15,093][44959] Updated weights for policy 1, policy_version 12060 (0.0007) [2023-10-12 20:27:15,907][44958] Updated weights for policy 0, policy_version 11970 (0.0009) [2023-10-12 20:27:16,292][44958] Updated weights for policy 0, policy_version 11980 (0.0009) [2023-10-12 20:27:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24608768. Throughput: 0: 1645.7, 1: 1643.6. Samples: 6165638. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-12 20:27:16,443][43579] Avg episode reward: [(0, '262.800'), (1, '254.910')] [2023-10-12 20:27:16,661][44958] Updated weights for policy 0, policy_version 11990 (0.0008) [2023-10-12 20:27:17,033][44958] Updated weights for policy 0, policy_version 12000 (0.0008) [2023-10-12 20:27:19,378][44959] Updated weights for policy 1, policy_version 12070 (0.0009) [2023-10-12 20:27:19,742][44959] Updated weights for policy 1, policy_version 12080 (0.0009) [2023-10-12 20:27:20,114][44959] Updated weights for policy 1, policy_version 12090 (0.0009) [2023-10-12 20:27:21,062][44958] Updated weights for policy 0, policy_version 12010 (0.0008) [2023-10-12 20:27:21,427][44958] Updated weights for policy 0, policy_version 12020 (0.0009) [2023-10-12 20:27:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24674304. Throughput: 0: 1637.6, 1: 1649.2. Samples: 6176188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:27:21,443][43579] Avg episode reward: [(0, '259.240'), (1, '255.190')] [2023-10-12 20:27:21,806][44958] Updated weights for policy 0, policy_version 12030 (0.0009) [2023-10-12 20:27:24,111][44959] Updated weights for policy 1, policy_version 12100 (0.0009) [2023-10-12 20:27:24,476][44959] Updated weights for policy 1, policy_version 12110 (0.0007) [2023-10-12 20:27:24,843][44959] Updated weights for policy 1, policy_version 12120 (0.0012) [2023-10-12 20:27:25,812][44958] Updated weights for policy 0, policy_version 12040 (0.0008) [2023-10-12 20:27:26,192][44958] Updated weights for policy 0, policy_version 12050 (0.0009) [2023-10-12 20:27:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24739840. Throughput: 0: 1640.8, 1: 1640.4. Samples: 6195456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:27:26,443][43579] Avg episode reward: [(0, '260.960'), (1, '254.480')] [2023-10-12 20:27:26,558][44958] Updated weights for policy 0, policy_version 12060 (0.0008) [2023-10-12 20:27:29,084][44959] Updated weights for policy 1, policy_version 12130 (0.0009) [2023-10-12 20:27:29,447][44959] Updated weights for policy 1, policy_version 12140 (0.0009) [2023-10-12 20:27:29,815][44959] Updated weights for policy 1, policy_version 12150 (0.0010) [2023-10-12 20:27:30,190][44959] Updated weights for policy 1, policy_version 12160 (0.0010) [2023-10-12 20:27:30,670][44958] Updated weights for policy 0, policy_version 12070 (0.0009) [2023-10-12 20:27:31,043][44958] Updated weights for policy 0, policy_version 12080 (0.0009) [2023-10-12 20:27:31,424][44958] Updated weights for policy 0, policy_version 12090 (0.0010) [2023-10-12 20:27:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24805376. Throughput: 0: 1645.6, 1: 1644.9. Samples: 6214892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:27:31,444][43579] Avg episode reward: [(0, '250.170'), (1, '264.640')] [2023-10-12 20:27:34,191][44959] Updated weights for policy 1, policy_version 12170 (0.0007) [2023-10-12 20:27:34,559][44959] Updated weights for policy 1, policy_version 12180 (0.0007) [2023-10-12 20:27:34,935][44959] Updated weights for policy 1, policy_version 12190 (0.0009) [2023-10-12 20:27:35,613][44958] Updated weights for policy 0, policy_version 12100 (0.0007) [2023-10-12 20:27:35,987][44958] Updated weights for policy 0, policy_version 12110 (0.0007) [2023-10-12 20:27:36,361][44958] Updated weights for policy 0, policy_version 12120 (0.0009) [2023-10-12 20:27:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24870912. Throughput: 0: 1648.5, 1: 1638.2. Samples: 6225400. Policy #0 lag: (min: 22.0, avg: 25.6, max: 54.0) [2023-10-12 20:27:36,443][43579] Avg episode reward: [(0, '255.700'), (1, '266.590')] [2023-10-12 20:27:39,093][44959] Updated weights for policy 1, policy_version 12200 (0.0010) [2023-10-12 20:27:39,458][44959] Updated weights for policy 1, policy_version 12210 (0.0010) [2023-10-12 20:27:39,832][44959] Updated weights for policy 1, policy_version 12220 (0.0010) [2023-10-12 20:27:40,579][44958] Updated weights for policy 0, policy_version 12130 (0.0009) [2023-10-12 20:27:40,956][44958] Updated weights for policy 0, policy_version 12140 (0.0007) [2023-10-12 20:27:41,329][44958] Updated weights for policy 0, policy_version 12150 (0.0009) [2023-10-12 20:27:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 24936448. Throughput: 0: 1642.0, 1: 1639.8. Samples: 6244682. Policy #0 lag: (min: 22.0, avg: 25.6, max: 54.0) [2023-10-12 20:27:41,443][43579] Avg episode reward: [(0, '259.040'), (1, '259.270')] [2023-10-12 20:27:41,696][44958] Updated weights for policy 0, policy_version 12160 (0.0008) [2023-10-12 20:27:44,112][44959] Updated weights for policy 1, policy_version 12230 (0.0010) [2023-10-12 20:27:44,480][44959] Updated weights for policy 1, policy_version 12240 (0.0009) [2023-10-12 20:27:44,837][44959] Updated weights for policy 1, policy_version 12250 (0.0011) [2023-10-12 20:27:45,836][44958] Updated weights for policy 0, policy_version 12170 (0.0007) [2023-10-12 20:27:46,217][44958] Updated weights for policy 0, policy_version 12180 (0.0009) [2023-10-12 20:27:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 25001984. Throughput: 0: 1645.7, 1: 1648.8. Samples: 6264348. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-12 20:27:46,444][43579] Avg episode reward: [(0, '258.810'), (1, '259.230')] [2023-10-12 20:27:46,456][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000012256_12550144.pth... [2023-10-12 20:27:46,490][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000010720_10977280.pth [2023-10-12 20:27:46,586][44958] Updated weights for policy 0, policy_version 12190 (0.0007) [2023-10-12 20:27:46,655][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000012192_12484608.pth... [2023-10-12 20:27:46,694][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000010656_10911744.pth [2023-10-12 20:27:49,017][44959] Updated weights for policy 1, policy_version 12260 (0.0009) [2023-10-12 20:27:49,422][44959] Updated weights for policy 1, policy_version 12270 (0.0009) [2023-10-12 20:27:49,783][44959] Updated weights for policy 1, policy_version 12280 (0.0010) [2023-10-12 20:27:50,754][44958] Updated weights for policy 0, policy_version 12200 (0.0008) [2023-10-12 20:27:51,125][44958] Updated weights for policy 0, policy_version 12210 (0.0007) [2023-10-12 20:27:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 25067520. Throughput: 0: 1644.3, 1: 1645.1. Samples: 6274780. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-12 20:27:51,443][43579] Avg episode reward: [(0, '260.690'), (1, '237.680')] [2023-10-12 20:27:51,497][44958] Updated weights for policy 0, policy_version 12220 (0.0009) [2023-10-12 20:27:53,945][44959] Updated weights for policy 1, policy_version 12290 (0.0011) [2023-10-12 20:27:54,311][44959] Updated weights for policy 1, policy_version 12300 (0.0010) [2023-10-12 20:27:54,674][44959] Updated weights for policy 1, policy_version 12310 (0.0011) [2023-10-12 20:27:55,038][44959] Updated weights for policy 1, policy_version 12320 (0.0011) [2023-10-12 20:27:55,624][44958] Updated weights for policy 0, policy_version 12230 (0.0009) [2023-10-12 20:27:55,996][44958] Updated weights for policy 0, policy_version 12240 (0.0007) [2023-10-12 20:27:56,377][44958] Updated weights for policy 0, policy_version 12250 (0.0010) [2023-10-12 20:27:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 25133056. Throughput: 0: 1647.6, 1: 1640.3. Samples: 6294192. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 20:27:56,443][43579] Avg episode reward: [(0, '260.960'), (1, '229.220')] [2023-10-12 20:27:59,071][44959] Updated weights for policy 1, policy_version 12330 (0.0008) [2023-10-12 20:27:59,442][44959] Updated weights for policy 1, policy_version 12340 (0.0008) [2023-10-12 20:27:59,811][44959] Updated weights for policy 1, policy_version 12350 (0.0007) [2023-10-12 20:28:00,514][44958] Updated weights for policy 0, policy_version 12260 (0.0008) [2023-10-12 20:28:00,903][44958] Updated weights for policy 0, policy_version 12270 (0.0007) [2023-10-12 20:28:01,286][44958] Updated weights for policy 0, policy_version 12280 (0.0007) [2023-10-12 20:28:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 25198592. Throughput: 0: 1642.9, 1: 1650.4. Samples: 6313834. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 20:28:01,443][43579] Avg episode reward: [(0, '254.550'), (1, '227.190')] [2023-10-12 20:28:03,901][44959] Updated weights for policy 1, policy_version 12360 (0.0007) [2023-10-12 20:28:04,269][44959] Updated weights for policy 1, policy_version 12370 (0.0011) [2023-10-12 20:28:04,645][44959] Updated weights for policy 1, policy_version 12380 (0.0010) [2023-10-12 20:28:05,522][44958] Updated weights for policy 0, policy_version 12290 (0.0008) [2023-10-12 20:28:05,888][44958] Updated weights for policy 0, policy_version 12300 (0.0010) [2023-10-12 20:28:06,263][44958] Updated weights for policy 0, policy_version 12310 (0.0008) [2023-10-12 20:28:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 25264128. Throughput: 0: 1652.4, 1: 1641.8. Samples: 6324428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:28:06,443][43579] Avg episode reward: [(0, '251.610'), (1, '232.460')] [2023-10-12 20:28:06,630][44958] Updated weights for policy 0, policy_version 12320 (0.0008) [2023-10-12 20:28:08,936][44959] Updated weights for policy 1, policy_version 12390 (0.0009) [2023-10-12 20:28:09,300][44959] Updated weights for policy 1, policy_version 12400 (0.0008) [2023-10-12 20:28:09,676][44959] Updated weights for policy 1, policy_version 12410 (0.0008) [2023-10-12 20:28:10,761][44958] Updated weights for policy 0, policy_version 12330 (0.0009) [2023-10-12 20:28:11,134][44958] Updated weights for policy 0, policy_version 12340 (0.0007) [2023-10-12 20:28:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 25329664. Throughput: 0: 1651.2, 1: 1642.7. Samples: 6343680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:28:11,443][43579] Avg episode reward: [(0, '250.110'), (1, '235.940')] [2023-10-12 20:28:11,500][44958] Updated weights for policy 0, policy_version 12350 (0.0009) [2023-10-12 20:28:13,797][44959] Updated weights for policy 1, policy_version 12420 (0.0009) [2023-10-12 20:28:14,171][44959] Updated weights for policy 1, policy_version 12430 (0.0010) [2023-10-12 20:28:14,535][44959] Updated weights for policy 1, policy_version 12440 (0.0008) [2023-10-12 20:28:15,649][44958] Updated weights for policy 0, policy_version 12360 (0.0010) [2023-10-12 20:28:16,018][44958] Updated weights for policy 0, policy_version 12370 (0.0008) [2023-10-12 20:28:16,394][44958] Updated weights for policy 0, policy_version 12380 (0.0008) [2023-10-12 20:28:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 25395200. Throughput: 0: 1645.4, 1: 1656.4. Samples: 6363472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:28:16,443][43579] Avg episode reward: [(0, '257.120'), (1, '244.990')] [2023-10-12 20:28:18,546][44959] Updated weights for policy 1, policy_version 12450 (0.0008) [2023-10-12 20:28:18,917][44959] Updated weights for policy 1, policy_version 12460 (0.0010) [2023-10-12 20:28:19,285][44959] Updated weights for policy 1, policy_version 12470 (0.0008) [2023-10-12 20:28:19,648][44959] Updated weights for policy 1, policy_version 12480 (0.0007) [2023-10-12 20:28:20,660][44958] Updated weights for policy 0, policy_version 12390 (0.0009) [2023-10-12 20:28:21,024][44958] Updated weights for policy 0, policy_version 12400 (0.0010) [2023-10-12 20:28:21,400][44958] Updated weights for policy 0, policy_version 12410 (0.0009) [2023-10-12 20:28:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 25460736. Throughput: 0: 1646.5, 1: 1653.2. Samples: 6373884. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:28:21,443][43579] Avg episode reward: [(0, '253.450'), (1, '258.380')] [2023-10-12 20:28:23,756][44959] Updated weights for policy 1, policy_version 12490 (0.0007) [2023-10-12 20:28:24,122][44959] Updated weights for policy 1, policy_version 12500 (0.0007) [2023-10-12 20:28:24,493][44959] Updated weights for policy 1, policy_version 12510 (0.0007) [2023-10-12 20:28:25,568][44958] Updated weights for policy 0, policy_version 12420 (0.0007) [2023-10-12 20:28:25,941][44958] Updated weights for policy 0, policy_version 12430 (0.0007) [2023-10-12 20:28:26,317][44958] Updated weights for policy 0, policy_version 12440 (0.0008) [2023-10-12 20:28:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25526272. Throughput: 0: 1650.7, 1: 1659.7. Samples: 6393650. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:28:26,443][43579] Avg episode reward: [(0, '259.880'), (1, '248.440')] [2023-10-12 20:28:28,422][44959] Updated weights for policy 1, policy_version 12520 (0.0009) [2023-10-12 20:28:28,791][44959] Updated weights for policy 1, policy_version 12530 (0.0010) [2023-10-12 20:28:29,159][44959] Updated weights for policy 1, policy_version 12540 (0.0009) [2023-10-12 20:28:30,404][44958] Updated weights for policy 0, policy_version 12450 (0.0009) [2023-10-12 20:28:30,776][44958] Updated weights for policy 0, policy_version 12460 (0.0007) [2023-10-12 20:28:31,142][44958] Updated weights for policy 0, policy_version 12470 (0.0007) [2023-10-12 20:28:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 25591808. Throughput: 0: 1635.7, 1: 1666.7. Samples: 6412956. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:28:31,443][43579] Avg episode reward: [(0, '262.420'), (1, '247.570')] [2023-10-12 20:28:31,521][44958] Updated weights for policy 0, policy_version 12480 (0.0008) [2023-10-12 20:28:33,453][44959] Updated weights for policy 1, policy_version 12550 (0.0008) [2023-10-12 20:28:33,818][44959] Updated weights for policy 1, policy_version 12560 (0.0008) [2023-10-12 20:28:34,189][44959] Updated weights for policy 1, policy_version 12570 (0.0007) [2023-10-12 20:28:35,773][44958] Updated weights for policy 0, policy_version 12490 (0.0007) [2023-10-12 20:28:36,141][44958] Updated weights for policy 0, policy_version 12500 (0.0008) [2023-10-12 20:28:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 25657344. Throughput: 0: 1643.9, 1: 1656.3. Samples: 6423288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:28:36,443][43579] Avg episode reward: [(0, '262.100'), (1, '248.510')] [2023-10-12 20:28:36,519][44958] Updated weights for policy 0, policy_version 12510 (0.0007) [2023-10-12 20:28:38,360][44959] Updated weights for policy 1, policy_version 12580 (0.0009) [2023-10-12 20:28:38,746][44959] Updated weights for policy 1, policy_version 12590 (0.0011) [2023-10-12 20:28:39,128][44959] Updated weights for policy 1, policy_version 12600 (0.0010) [2023-10-12 20:28:40,769][44958] Updated weights for policy 0, policy_version 12520 (0.0010) [2023-10-12 20:28:41,144][44958] Updated weights for policy 0, policy_version 12530 (0.0011) [2023-10-12 20:28:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 25722880. Throughput: 0: 1639.2, 1: 1669.1. Samples: 6443070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:28:41,444][43579] Avg episode reward: [(0, '253.480'), (1, '245.140')] [2023-10-12 20:28:41,519][44958] Updated weights for policy 0, policy_version 12540 (0.0011) [2023-10-12 20:28:42,987][44959] Updated weights for policy 1, policy_version 12610 (0.0008) [2023-10-12 20:28:43,364][44959] Updated weights for policy 1, policy_version 12620 (0.0009) [2023-10-12 20:28:43,724][44959] Updated weights for policy 1, policy_version 12630 (0.0008) [2023-10-12 20:28:44,099][44959] Updated weights for policy 1, policy_version 12640 (0.0008) [2023-10-12 20:28:45,745][44958] Updated weights for policy 0, policy_version 12550 (0.0008) [2023-10-12 20:28:46,130][44958] Updated weights for policy 0, policy_version 12560 (0.0008) [2023-10-12 20:28:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 25788416. Throughput: 0: 1638.9, 1: 1675.8. Samples: 6462996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:28:46,443][43579] Avg episode reward: [(0, '258.210'), (1, '251.290')] [2023-10-12 20:28:46,507][44958] Updated weights for policy 0, policy_version 12570 (0.0009) [2023-10-12 20:28:48,313][44959] Updated weights for policy 1, policy_version 12650 (0.0010) [2023-10-12 20:28:48,683][44959] Updated weights for policy 1, policy_version 12660 (0.0010) [2023-10-12 20:28:49,052][44959] Updated weights for policy 1, policy_version 12670 (0.0010) [2023-10-12 20:28:50,678][44958] Updated weights for policy 0, policy_version 12580 (0.0007) [2023-10-12 20:28:51,055][44958] Updated weights for policy 0, policy_version 12590 (0.0007) [2023-10-12 20:28:51,428][44958] Updated weights for policy 0, policy_version 12600 (0.0007) [2023-10-12 20:28:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 25853952. Throughput: 0: 1632.7, 1: 1658.0. Samples: 6472510. Policy #0 lag: (min: 12.0, avg: 19.4, max: 44.0) [2023-10-12 20:28:51,444][43579] Avg episode reward: [(0, '258.310'), (1, '248.770')] [2023-10-12 20:28:53,354][44959] Updated weights for policy 1, policy_version 12680 (0.0007) [2023-10-12 20:28:53,722][44959] Updated weights for policy 1, policy_version 12690 (0.0009) [2023-10-12 20:28:54,095][44959] Updated weights for policy 1, policy_version 12700 (0.0009) [2023-10-12 20:28:55,480][44958] Updated weights for policy 0, policy_version 12610 (0.0007) [2023-10-12 20:28:55,859][44958] Updated weights for policy 0, policy_version 12620 (0.0009) [2023-10-12 20:28:56,221][44958] Updated weights for policy 0, policy_version 12630 (0.0008) [2023-10-12 20:28:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 25919488. Throughput: 0: 1636.3, 1: 1672.4. Samples: 6492572. Policy #0 lag: (min: 12.0, avg: 19.4, max: 44.0) [2023-10-12 20:28:56,444][43579] Avg episode reward: [(0, '252.630'), (1, '251.510')] [2023-10-12 20:28:56,592][44958] Updated weights for policy 0, policy_version 12640 (0.0007) [2023-10-12 20:28:58,029][44959] Updated weights for policy 1, policy_version 12710 (0.0010) [2023-10-12 20:28:58,386][44959] Updated weights for policy 1, policy_version 12720 (0.0010) [2023-10-12 20:28:58,764][44959] Updated weights for policy 1, policy_version 12730 (0.0008) [2023-10-12 20:29:00,859][44958] Updated weights for policy 0, policy_version 12650 (0.0007) [2023-10-12 20:29:01,235][44958] Updated weights for policy 0, policy_version 12660 (0.0007) [2023-10-12 20:29:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 25985024. Throughput: 0: 1639.7, 1: 1668.3. Samples: 6512334. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 20:29:01,444][43579] Avg episode reward: [(0, '254.080'), (1, '248.800')] [2023-10-12 20:29:01,621][44958] Updated weights for policy 0, policy_version 12670 (0.0008) [2023-10-12 20:29:02,808][44959] Updated weights for policy 1, policy_version 12740 (0.0008) [2023-10-12 20:29:03,187][44959] Updated weights for policy 1, policy_version 12750 (0.0007) [2023-10-12 20:29:03,553][44959] Updated weights for policy 1, policy_version 12760 (0.0007) [2023-10-12 20:29:05,611][44958] Updated weights for policy 0, policy_version 12680 (0.0008) [2023-10-12 20:29:05,978][44958] Updated weights for policy 0, policy_version 12690 (0.0007) [2023-10-12 20:29:06,348][44958] Updated weights for policy 0, policy_version 12700 (0.0009) [2023-10-12 20:29:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 26050560. Throughput: 0: 1642.4, 1: 1650.6. Samples: 6522066. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 20:29:06,443][43579] Avg episode reward: [(0, '258.230'), (1, '241.270')] [2023-10-12 20:29:07,806][44959] Updated weights for policy 1, policy_version 12770 (0.0007) [2023-10-12 20:29:08,165][44959] Updated weights for policy 1, policy_version 12780 (0.0009) [2023-10-12 20:29:08,538][44959] Updated weights for policy 1, policy_version 12790 (0.0010) [2023-10-12 20:29:08,914][44959] Updated weights for policy 1, policy_version 12800 (0.0009) [2023-10-12 20:29:10,490][44958] Updated weights for policy 0, policy_version 12710 (0.0007) [2023-10-12 20:29:10,867][44958] Updated weights for policy 0, policy_version 12720 (0.0007) [2023-10-12 20:29:11,243][44958] Updated weights for policy 0, policy_version 12730 (0.0007) [2023-10-12 20:29:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 26116096. Throughput: 0: 1637.8, 1: 1660.7. Samples: 6542082. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 20:29:11,443][43579] Avg episode reward: [(0, '259.140'), (1, '243.970')] [2023-10-12 20:29:13,046][44959] Updated weights for policy 1, policy_version 12810 (0.0009) [2023-10-12 20:29:13,411][44959] Updated weights for policy 1, policy_version 12820 (0.0010) [2023-10-12 20:29:13,771][44959] Updated weights for policy 1, policy_version 12830 (0.0010) [2023-10-12 20:29:15,520][44958] Updated weights for policy 0, policy_version 12740 (0.0009) [2023-10-12 20:29:15,900][44958] Updated weights for policy 0, policy_version 12750 (0.0007) [2023-10-12 20:29:16,270][44958] Updated weights for policy 0, policy_version 12760 (0.0007) [2023-10-12 20:29:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 26181632. Throughput: 0: 1643.3, 1: 1662.4. Samples: 6561714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:29:16,443][43579] Avg episode reward: [(0, '253.260'), (1, '243.970')] [2023-10-12 20:29:17,714][44959] Updated weights for policy 1, policy_version 12840 (0.0008) [2023-10-12 20:29:18,085][44959] Updated weights for policy 1, policy_version 12850 (0.0008) [2023-10-12 20:29:18,452][44959] Updated weights for policy 1, policy_version 12860 (0.0008) [2023-10-12 20:29:20,244][44958] Updated weights for policy 0, policy_version 12770 (0.0007) [2023-10-12 20:29:20,619][44958] Updated weights for policy 0, policy_version 12780 (0.0008) [2023-10-12 20:29:20,993][44958] Updated weights for policy 0, policy_version 12790 (0.0008) [2023-10-12 20:29:21,365][44958] Updated weights for policy 0, policy_version 12800 (0.0007) [2023-10-12 20:29:21,443][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 26279936. Throughput: 0: 1644.6, 1: 1650.7. Samples: 6571576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:29:21,443][43579] Avg episode reward: [(0, '256.990'), (1, '234.890')] [2023-10-12 20:29:22,800][44959] Updated weights for policy 1, policy_version 12870 (0.0009) [2023-10-12 20:29:23,168][44959] Updated weights for policy 1, policy_version 12880 (0.0010) [2023-10-12 20:29:23,534][44959] Updated weights for policy 1, policy_version 12890 (0.0010) [2023-10-12 20:29:25,632][44958] Updated weights for policy 0, policy_version 12810 (0.0010) [2023-10-12 20:29:26,009][44958] Updated weights for policy 0, policy_version 12820 (0.0008) [2023-10-12 20:29:26,388][44958] Updated weights for policy 0, policy_version 12830 (0.0008) [2023-10-12 20:29:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 26312704. Throughput: 0: 1643.9, 1: 1655.9. Samples: 6591560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:29:26,444][43579] Avg episode reward: [(0, '257.400'), (1, '229.300')] [2023-10-12 20:29:27,840][44959] Updated weights for policy 1, policy_version 12900 (0.0008) [2023-10-12 20:29:28,237][44959] Updated weights for policy 1, policy_version 12910 (0.0011) [2023-10-12 20:29:28,608][44959] Updated weights for policy 1, policy_version 12920 (0.0008) [2023-10-12 20:29:30,584][44958] Updated weights for policy 0, policy_version 12840 (0.0009) [2023-10-12 20:29:30,955][44958] Updated weights for policy 0, policy_version 12850 (0.0010) [2023-10-12 20:29:31,327][44958] Updated weights for policy 0, policy_version 12860 (0.0008) [2023-10-12 20:29:31,442][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 26378240. Throughput: 0: 1643.4, 1: 1653.1. Samples: 6611340. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-12 20:29:31,443][43579] Avg episode reward: [(0, '257.790'), (1, '227.330')] [2023-10-12 20:29:32,583][44959] Updated weights for policy 1, policy_version 12930 (0.0007) [2023-10-12 20:29:32,949][44959] Updated weights for policy 1, policy_version 12940 (0.0007) [2023-10-12 20:29:33,324][44959] Updated weights for policy 1, policy_version 12950 (0.0009) [2023-10-12 20:29:33,691][44959] Updated weights for policy 1, policy_version 12960 (0.0010) [2023-10-12 20:29:35,465][44958] Updated weights for policy 0, policy_version 12870 (0.0009) [2023-10-12 20:29:35,845][44958] Updated weights for policy 0, policy_version 12880 (0.0010) [2023-10-12 20:29:36,211][44958] Updated weights for policy 0, policy_version 12890 (0.0010) [2023-10-12 20:29:36,442][43579] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 26476544. Throughput: 0: 1646.3, 1: 1649.2. Samples: 6620804. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-12 20:29:36,443][43579] Avg episode reward: [(0, '256.640'), (1, '226.160')] [2023-10-12 20:29:37,879][44959] Updated weights for policy 1, policy_version 12970 (0.0009) [2023-10-12 20:29:38,250][44959] Updated weights for policy 1, policy_version 12980 (0.0007) [2023-10-12 20:29:38,629][44959] Updated weights for policy 1, policy_version 12990 (0.0008) [2023-10-12 20:29:40,522][44958] Updated weights for policy 0, policy_version 12900 (0.0008) [2023-10-12 20:29:40,895][44958] Updated weights for policy 0, policy_version 12910 (0.0007) [2023-10-12 20:29:41,269][44958] Updated weights for policy 0, policy_version 12920 (0.0007) [2023-10-12 20:29:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 26509312. Throughput: 0: 1640.8, 1: 1653.6. Samples: 6640818. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-12 20:29:41,443][43579] Avg episode reward: [(0, '265.900'), (1, '225.850')] [2023-10-12 20:29:42,506][44959] Updated weights for policy 1, policy_version 13000 (0.0009) [2023-10-12 20:29:42,880][44959] Updated weights for policy 1, policy_version 13010 (0.0011) [2023-10-12 20:29:43,249][44959] Updated weights for policy 1, policy_version 13020 (0.0010) [2023-10-12 20:29:45,224][44958] Updated weights for policy 0, policy_version 12930 (0.0008) [2023-10-12 20:29:45,597][44958] Updated weights for policy 0, policy_version 12940 (0.0009) [2023-10-12 20:29:45,971][44958] Updated weights for policy 0, policy_version 12950 (0.0007) [2023-10-12 20:29:46,335][44958] Updated weights for policy 0, policy_version 12960 (0.0011) [2023-10-12 20:29:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 26607616. Throughput: 0: 1633.9, 1: 1650.1. Samples: 6660114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:29:46,443][43579] Avg episode reward: [(0, '270.150'), (1, '231.360')] [2023-10-12 20:29:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000013024_13336576.pth... [2023-10-12 20:29:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000012960_13271040.pth... [2023-10-12 20:29:46,482][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000011488_11763712.pth [2023-10-12 20:29:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000011424_11698176.pth [2023-10-12 20:29:47,659][44959] Updated weights for policy 1, policy_version 13030 (0.0009) [2023-10-12 20:29:48,035][44959] Updated weights for policy 1, policy_version 13040 (0.0008) [2023-10-12 20:29:48,398][44959] Updated weights for policy 1, policy_version 13050 (0.0008) [2023-10-12 20:29:50,679][44958] Updated weights for policy 0, policy_version 12970 (0.0007) [2023-10-12 20:29:51,054][44958] Updated weights for policy 0, policy_version 12980 (0.0007) [2023-10-12 20:29:51,430][44958] Updated weights for policy 0, policy_version 12990 (0.0007) [2023-10-12 20:29:51,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 26640384. Throughput: 0: 1634.6, 1: 1646.3. Samples: 6669706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:29:51,444][43579] Avg episode reward: [(0, '268.220'), (1, '237.140')] [2023-10-12 20:29:52,621][44959] Updated weights for policy 1, policy_version 13060 (0.0009) [2023-10-12 20:29:52,989][44959] Updated weights for policy 1, policy_version 13070 (0.0009) [2023-10-12 20:29:53,363][44959] Updated weights for policy 1, policy_version 13080 (0.0007) [2023-10-12 20:29:55,524][44958] Updated weights for policy 0, policy_version 13000 (0.0007) [2023-10-12 20:29:55,907][44958] Updated weights for policy 0, policy_version 13010 (0.0011) [2023-10-12 20:29:56,275][44958] Updated weights for policy 0, policy_version 13020 (0.0007) [2023-10-12 20:29:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 26738688. Throughput: 0: 1639.2, 1: 1649.2. Samples: 6690060. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-12 20:29:56,443][43579] Avg episode reward: [(0, '260.970'), (1, '250.680')] [2023-10-12 20:29:57,576][44959] Updated weights for policy 1, policy_version 13090 (0.0009) [2023-10-12 20:29:57,950][44959] Updated weights for policy 1, policy_version 13100 (0.0008) [2023-10-12 20:29:58,315][44959] Updated weights for policy 1, policy_version 13110 (0.0008) [2023-10-12 20:29:58,681][44959] Updated weights for policy 1, policy_version 13120 (0.0007) [2023-10-12 20:30:00,309][44958] Updated weights for policy 0, policy_version 13030 (0.0009) [2023-10-12 20:30:00,689][44958] Updated weights for policy 0, policy_version 13040 (0.0008) [2023-10-12 20:30:01,066][44958] Updated weights for policy 0, policy_version 13050 (0.0007) [2023-10-12 20:30:01,443][43579] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 26804224. Throughput: 0: 1639.3, 1: 1647.4. Samples: 6709616. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-12 20:30:01,444][43579] Avg episode reward: [(0, '259.670'), (1, '244.450')] [2023-10-12 20:30:02,865][44959] Updated weights for policy 1, policy_version 13130 (0.0008) [2023-10-12 20:30:03,227][44959] Updated weights for policy 1, policy_version 13140 (0.0007) [2023-10-12 20:30:03,596][44959] Updated weights for policy 1, policy_version 13150 (0.0010) [2023-10-12 20:30:05,334][44958] Updated weights for policy 0, policy_version 13060 (0.0007) [2023-10-12 20:30:05,712][44958] Updated weights for policy 0, policy_version 13070 (0.0007) [2023-10-12 20:30:06,094][44958] Updated weights for policy 0, policy_version 13080 (0.0010) [2023-10-12 20:30:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 26869760. Throughput: 0: 1642.0, 1: 1644.4. Samples: 6719466. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-12 20:30:06,443][43579] Avg episode reward: [(0, '260.140'), (1, '245.400')] [2023-10-12 20:30:07,780][44959] Updated weights for policy 1, policy_version 13160 (0.0009) [2023-10-12 20:30:08,141][44959] Updated weights for policy 1, policy_version 13170 (0.0011) [2023-10-12 20:30:08,508][44959] Updated weights for policy 1, policy_version 13180 (0.0008) [2023-10-12 20:30:10,154][44958] Updated weights for policy 0, policy_version 13090 (0.0009) [2023-10-12 20:30:10,525][44958] Updated weights for policy 0, policy_version 13100 (0.0009) [2023-10-12 20:30:10,903][44958] Updated weights for policy 0, policy_version 13110 (0.0009) [2023-10-12 20:30:11,270][44958] Updated weights for policy 0, policy_version 13120 (0.0009) [2023-10-12 20:30:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 26935296. Throughput: 0: 1646.8, 1: 1651.2. Samples: 6739974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:30:11,443][43579] Avg episode reward: [(0, '249.360'), (1, '239.420')] [2023-10-12 20:30:12,605][44959] Updated weights for policy 1, policy_version 13190 (0.0009) [2023-10-12 20:30:12,978][44959] Updated weights for policy 1, policy_version 13200 (0.0007) [2023-10-12 20:30:13,340][44959] Updated weights for policy 1, policy_version 13210 (0.0007) [2023-10-12 20:30:15,548][44958] Updated weights for policy 0, policy_version 13130 (0.0008) [2023-10-12 20:30:15,920][44958] Updated weights for policy 0, policy_version 13140 (0.0008) [2023-10-12 20:30:16,299][44958] Updated weights for policy 0, policy_version 13150 (0.0008) [2023-10-12 20:30:16,443][43579] Fps is (10 sec: 13106.6, 60 sec: 13653.2, 300 sec: 13218.3). Total num frames: 27000832. Throughput: 0: 1640.2, 1: 1647.3. Samples: 6759276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:30:16,444][43579] Avg episode reward: [(0, '253.710'), (1, '241.540')] [2023-10-12 20:30:17,475][44959] Updated weights for policy 1, policy_version 13220 (0.0010) [2023-10-12 20:30:17,846][44959] Updated weights for policy 1, policy_version 13230 (0.0007) [2023-10-12 20:30:18,213][44959] Updated weights for policy 1, policy_version 13240 (0.0007) [2023-10-12 20:30:20,677][44958] Updated weights for policy 0, policy_version 13160 (0.0008) [2023-10-12 20:30:21,059][44958] Updated weights for policy 0, policy_version 13170 (0.0007) [2023-10-12 20:30:21,429][44958] Updated weights for policy 0, policy_version 13180 (0.0008) [2023-10-12 20:30:21,442][43579] Fps is (10 sec: 9830.3, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 27033600. Throughput: 0: 1642.8, 1: 1647.6. Samples: 6768870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:30:21,443][43579] Avg episode reward: [(0, '258.320'), (1, '240.270')] [2023-10-12 20:30:22,333][44959] Updated weights for policy 1, policy_version 13250 (0.0009) [2023-10-12 20:30:22,699][44959] Updated weights for policy 1, policy_version 13260 (0.0008) [2023-10-12 20:30:23,059][44959] Updated weights for policy 1, policy_version 13270 (0.0009) [2023-10-12 20:30:23,431][44959] Updated weights for policy 1, policy_version 13280 (0.0008) [2023-10-12 20:30:25,463][44958] Updated weights for policy 0, policy_version 13190 (0.0009) [2023-10-12 20:30:25,841][44958] Updated weights for policy 0, policy_version 13200 (0.0008) [2023-10-12 20:30:26,209][44958] Updated weights for policy 0, policy_version 13210 (0.0008) [2023-10-12 20:30:26,442][43579] Fps is (10 sec: 13107.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 27131904. Throughput: 0: 1647.0, 1: 1653.8. Samples: 6789354. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 20:30:26,443][43579] Avg episode reward: [(0, '262.400'), (1, '251.950')] [2023-10-12 20:30:27,483][44959] Updated weights for policy 1, policy_version 13290 (0.0008) [2023-10-12 20:30:27,855][44959] Updated weights for policy 1, policy_version 13300 (0.0007) [2023-10-12 20:30:28,212][44959] Updated weights for policy 1, policy_version 13310 (0.0011) [2023-10-12 20:30:30,275][44958] Updated weights for policy 0, policy_version 13220 (0.0008) [2023-10-12 20:30:30,636][44958] Updated weights for policy 0, policy_version 13230 (0.0008) [2023-10-12 20:30:31,010][44958] Updated weights for policy 0, policy_version 13240 (0.0007) [2023-10-12 20:30:31,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 27197440. Throughput: 0: 1646.0, 1: 1656.8. Samples: 6808736. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 20:30:31,444][43579] Avg episode reward: [(0, '264.490'), (1, '248.940')] [2023-10-12 20:30:32,383][44959] Updated weights for policy 1, policy_version 13320 (0.0008) [2023-10-12 20:30:32,756][44959] Updated weights for policy 1, policy_version 13330 (0.0010) [2023-10-12 20:30:33,130][44959] Updated weights for policy 1, policy_version 13340 (0.0008) [2023-10-12 20:30:35,243][44958] Updated weights for policy 0, policy_version 13250 (0.0007) [2023-10-12 20:30:35,619][44958] Updated weights for policy 0, policy_version 13260 (0.0007) [2023-10-12 20:30:35,998][44958] Updated weights for policy 0, policy_version 13270 (0.0008) [2023-10-12 20:30:36,372][44958] Updated weights for policy 0, policy_version 13280 (0.0010) [2023-10-12 20:30:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 27262976. Throughput: 0: 1652.9, 1: 1656.7. Samples: 6818636. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 20:30:36,443][43579] Avg episode reward: [(0, '267.820'), (1, '249.300')] [2023-10-12 20:30:37,251][44959] Updated weights for policy 1, policy_version 13350 (0.0010) [2023-10-12 20:30:37,623][44959] Updated weights for policy 1, policy_version 13360 (0.0009) [2023-10-12 20:30:37,983][44959] Updated weights for policy 1, policy_version 13370 (0.0011) [2023-10-12 20:30:40,582][44958] Updated weights for policy 0, policy_version 13290 (0.0008) [2023-10-12 20:30:40,958][44958] Updated weights for policy 0, policy_version 13300 (0.0009) [2023-10-12 20:30:41,329][44958] Updated weights for policy 0, policy_version 13310 (0.0008) [2023-10-12 20:30:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 27328512. Throughput: 0: 1654.7, 1: 1656.0. Samples: 6839042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:30:41,443][43579] Avg episode reward: [(0, '269.860'), (1, '250.770')] [2023-10-12 20:30:42,276][44959] Updated weights for policy 1, policy_version 13380 (0.0011) [2023-10-12 20:30:42,647][44959] Updated weights for policy 1, policy_version 13390 (0.0008) [2023-10-12 20:30:43,015][44959] Updated weights for policy 1, policy_version 13400 (0.0010) [2023-10-12 20:30:45,412][44958] Updated weights for policy 0, policy_version 13320 (0.0010) [2023-10-12 20:30:45,791][44958] Updated weights for policy 0, policy_version 13330 (0.0007) [2023-10-12 20:30:46,159][44958] Updated weights for policy 0, policy_version 13340 (0.0008) [2023-10-12 20:30:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27394048. Throughput: 0: 1648.8, 1: 1655.1. Samples: 6858294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:30:46,444][43579] Avg episode reward: [(0, '268.070'), (1, '256.590')] [2023-10-12 20:30:47,129][44959] Updated weights for policy 1, policy_version 13410 (0.0008) [2023-10-12 20:30:47,493][44959] Updated weights for policy 1, policy_version 13420 (0.0007) [2023-10-12 20:30:47,866][44959] Updated weights for policy 1, policy_version 13430 (0.0007) [2023-10-12 20:30:48,229][44959] Updated weights for policy 1, policy_version 13440 (0.0010) [2023-10-12 20:30:50,550][44958] Updated weights for policy 0, policy_version 13350 (0.0008) [2023-10-12 20:30:50,928][44958] Updated weights for policy 0, policy_version 13360 (0.0009) [2023-10-12 20:30:51,297][44958] Updated weights for policy 0, policy_version 13370 (0.0007) [2023-10-12 20:30:51,443][43579] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 27426816. Throughput: 0: 1643.0, 1: 1653.7. Samples: 6867820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:30:51,444][43579] Avg episode reward: [(0, '267.390'), (1, '258.740')] [2023-10-12 20:30:52,390][44959] Updated weights for policy 1, policy_version 13450 (0.0009) [2023-10-12 20:30:52,767][44959] Updated weights for policy 1, policy_version 13460 (0.0008) [2023-10-12 20:30:53,133][44959] Updated weights for policy 1, policy_version 13470 (0.0010) [2023-10-12 20:30:55,438][44958] Updated weights for policy 0, policy_version 13380 (0.0007) [2023-10-12 20:30:55,800][44958] Updated weights for policy 0, policy_version 13390 (0.0007) [2023-10-12 20:30:56,177][44958] Updated weights for policy 0, policy_version 13400 (0.0007) [2023-10-12 20:30:56,442][43579] Fps is (10 sec: 9830.7, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 27492352. Throughput: 0: 1640.8, 1: 1650.9. Samples: 6888102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:30:56,443][43579] Avg episode reward: [(0, '266.870'), (1, '254.110')] [2023-10-12 20:30:57,372][44959] Updated weights for policy 1, policy_version 13480 (0.0007) [2023-10-12 20:30:57,735][44959] Updated weights for policy 1, policy_version 13490 (0.0008) [2023-10-12 20:30:58,108][44959] Updated weights for policy 1, policy_version 13500 (0.0010) [2023-10-12 20:31:00,273][44958] Updated weights for policy 0, policy_version 13410 (0.0007) [2023-10-12 20:31:00,675][44958] Updated weights for policy 0, policy_version 13420 (0.0007) [2023-10-12 20:31:01,042][44958] Updated weights for policy 0, policy_version 13430 (0.0008) [2023-10-12 20:31:01,406][44958] Updated weights for policy 0, policy_version 13440 (0.0009) [2023-10-12 20:31:01,443][43579] Fps is (10 sec: 16384.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27590656. Throughput: 0: 1641.4, 1: 1655.6. Samples: 6907640. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 20:31:01,443][43579] Avg episode reward: [(0, '263.990'), (1, '257.720')] [2023-10-12 20:31:02,158][44959] Updated weights for policy 1, policy_version 13510 (0.0009) [2023-10-12 20:31:02,548][44959] Updated weights for policy 1, policy_version 13520 (0.0010) [2023-10-12 20:31:02,918][44959] Updated weights for policy 1, policy_version 13530 (0.0010) [2023-10-12 20:31:05,404][44958] Updated weights for policy 0, policy_version 13450 (0.0008) [2023-10-12 20:31:05,769][44958] Updated weights for policy 0, policy_version 13460 (0.0008) [2023-10-12 20:31:06,139][44958] Updated weights for policy 0, policy_version 13470 (0.0010) [2023-10-12 20:31:06,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27656192. Throughput: 0: 1646.1, 1: 1653.9. Samples: 6917370. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 20:31:06,443][43579] Avg episode reward: [(0, '265.730'), (1, '261.030')] [2023-10-12 20:31:06,999][44959] Updated weights for policy 1, policy_version 13540 (0.0009) [2023-10-12 20:31:07,359][44959] Updated weights for policy 1, policy_version 13550 (0.0008) [2023-10-12 20:31:07,725][44959] Updated weights for policy 1, policy_version 13560 (0.0008) [2023-10-12 20:31:10,224][44958] Updated weights for policy 0, policy_version 13480 (0.0009) [2023-10-12 20:31:10,594][44958] Updated weights for policy 0, policy_version 13490 (0.0007) [2023-10-12 20:31:10,969][44958] Updated weights for policy 0, policy_version 13500 (0.0008) [2023-10-12 20:31:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 27721728. Throughput: 0: 1639.5, 1: 1649.5. Samples: 6937364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:31:11,444][43579] Avg episode reward: [(0, '268.900'), (1, '247.460')] [2023-10-12 20:31:11,859][44959] Updated weights for policy 1, policy_version 13570 (0.0009) [2023-10-12 20:31:12,224][44959] Updated weights for policy 1, policy_version 13580 (0.0008) [2023-10-12 20:31:12,591][44959] Updated weights for policy 1, policy_version 13590 (0.0008) [2023-10-12 20:31:12,963][44959] Updated weights for policy 1, policy_version 13600 (0.0007) [2023-10-12 20:31:15,273][44958] Updated weights for policy 0, policy_version 13510 (0.0009) [2023-10-12 20:31:15,648][44958] Updated weights for policy 0, policy_version 13520 (0.0009) [2023-10-12 20:31:16,031][44958] Updated weights for policy 0, policy_version 13530 (0.0009) [2023-10-12 20:31:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 27787264. Throughput: 0: 1640.7, 1: 1653.9. Samples: 6956990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:31:16,443][43579] Avg episode reward: [(0, '268.250'), (1, '234.960')] [2023-10-12 20:31:16,972][44959] Updated weights for policy 1, policy_version 13610 (0.0009) [2023-10-12 20:31:17,347][44959] Updated weights for policy 1, policy_version 13620 (0.0007) [2023-10-12 20:31:17,712][44959] Updated weights for policy 1, policy_version 13630 (0.0007) [2023-10-12 20:31:20,414][44958] Updated weights for policy 0, policy_version 13540 (0.0008) [2023-10-12 20:31:20,785][44958] Updated weights for policy 0, policy_version 13550 (0.0007) [2023-10-12 20:31:21,161][44958] Updated weights for policy 0, policy_version 13560 (0.0008) [2023-10-12 20:31:21,442][43579] Fps is (10 sec: 9830.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 27820032. Throughput: 0: 1637.0, 1: 1655.0. Samples: 6966776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:31:21,443][43579] Avg episode reward: [(0, '265.730'), (1, '238.090')] [2023-10-12 20:31:21,871][44959] Updated weights for policy 1, policy_version 13640 (0.0008) [2023-10-12 20:31:22,232][44959] Updated weights for policy 1, policy_version 13650 (0.0007) [2023-10-12 20:31:22,598][44959] Updated weights for policy 1, policy_version 13660 (0.0007) [2023-10-12 20:31:25,325][44958] Updated weights for policy 0, policy_version 13570 (0.0008) [2023-10-12 20:31:25,709][44958] Updated weights for policy 0, policy_version 13580 (0.0009) [2023-10-12 20:31:26,072][44958] Updated weights for policy 0, policy_version 13590 (0.0010) [2023-10-12 20:31:26,443][43579] Fps is (10 sec: 9830.3, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 27885568. Throughput: 0: 1634.9, 1: 1656.7. Samples: 6987168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:31:26,444][43579] Avg episode reward: [(0, '266.480'), (1, '240.060')] [2023-10-12 20:31:26,451][44958] Updated weights for policy 0, policy_version 13600 (0.0010) [2023-10-12 20:31:26,646][44959] Updated weights for policy 1, policy_version 13670 (0.0008) [2023-10-12 20:31:27,016][44959] Updated weights for policy 1, policy_version 13680 (0.0007) [2023-10-12 20:31:27,386][44959] Updated weights for policy 1, policy_version 13690 (0.0008) [2023-10-12 20:31:30,650][44958] Updated weights for policy 0, policy_version 13610 (0.0008) [2023-10-12 20:31:31,023][44958] Updated weights for policy 0, policy_version 13620 (0.0008) [2023-10-12 20:31:31,403][44958] Updated weights for policy 0, policy_version 13630 (0.0008) [2023-10-12 20:31:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 27951104. Throughput: 0: 1635.4, 1: 1660.0. Samples: 7006586. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 20:31:31,443][43579] Avg episode reward: [(0, '268.840'), (1, '235.760')] [2023-10-12 20:31:31,686][44959] Updated weights for policy 1, policy_version 13700 (0.0008) [2023-10-12 20:31:32,053][44959] Updated weights for policy 1, policy_version 13710 (0.0007) [2023-10-12 20:31:32,423][44959] Updated weights for policy 1, policy_version 13720 (0.0007) [2023-10-12 20:31:35,415][44958] Updated weights for policy 0, policy_version 13640 (0.0009) [2023-10-12 20:31:35,792][44958] Updated weights for policy 0, policy_version 13650 (0.0008) [2023-10-12 20:31:36,174][44958] Updated weights for policy 0, policy_version 13660 (0.0007) [2023-10-12 20:31:36,443][43579] Fps is (10 sec: 16384.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28049408. Throughput: 0: 1636.8, 1: 1664.0. Samples: 7016358. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 20:31:36,444][43579] Avg episode reward: [(0, '266.100'), (1, '234.650')] [2023-10-12 20:31:36,570][44959] Updated weights for policy 1, policy_version 13730 (0.0007) [2023-10-12 20:31:36,937][44959] Updated weights for policy 1, policy_version 13740 (0.0010) [2023-10-12 20:31:37,319][44959] Updated weights for policy 1, policy_version 13750 (0.0008) [2023-10-12 20:31:37,688][44959] Updated weights for policy 1, policy_version 13760 (0.0009) [2023-10-12 20:31:40,194][44958] Updated weights for policy 0, policy_version 13670 (0.0010) [2023-10-12 20:31:40,564][44958] Updated weights for policy 0, policy_version 13680 (0.0010) [2023-10-12 20:31:40,930][44958] Updated weights for policy 0, policy_version 13690 (0.0010) [2023-10-12 20:31:41,443][43579] Fps is (10 sec: 16384.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28114944. Throughput: 0: 1630.1, 1: 1662.6. Samples: 7036274. Policy #0 lag: (min: 21.0, avg: 21.5, max: 36.0) [2023-10-12 20:31:41,443][43579] Avg episode reward: [(0, '264.820'), (1, '242.910')] [2023-10-12 20:31:41,946][44959] Updated weights for policy 1, policy_version 13770 (0.0010) [2023-10-12 20:31:42,319][44959] Updated weights for policy 1, policy_version 13780 (0.0009) [2023-10-12 20:31:42,691][44959] Updated weights for policy 1, policy_version 13790 (0.0007) [2023-10-12 20:31:45,248][44958] Updated weights for policy 0, policy_version 13700 (0.0009) [2023-10-12 20:31:45,643][44958] Updated weights for policy 0, policy_version 13710 (0.0010) [2023-10-12 20:31:46,021][44958] Updated weights for policy 0, policy_version 13720 (0.0009) [2023-10-12 20:31:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28180480. Throughput: 0: 1632.0, 1: 1655.1. Samples: 7055558. Policy #0 lag: (min: 21.0, avg: 21.5, max: 36.0) [2023-10-12 20:31:46,443][43579] Avg episode reward: [(0, '271.120'), (1, '248.320')] [2023-10-12 20:31:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000013728_14057472.pth... [2023-10-12 20:31:46,483][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000012192_12484608.pth [2023-10-12 20:31:46,915][44959] Updated weights for policy 1, policy_version 13800 (0.0008) [2023-10-12 20:31:47,296][44959] Updated weights for policy 1, policy_version 13810 (0.0009) [2023-10-12 20:31:47,668][44959] Updated weights for policy 1, policy_version 13820 (0.0010) [2023-10-12 20:31:47,815][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000013824_14155776.pth... [2023-10-12 20:31:47,854][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000012256_12550144.pth [2023-10-12 20:31:50,111][44958] Updated weights for policy 0, policy_version 13730 (0.0007) [2023-10-12 20:31:50,496][44958] Updated weights for policy 0, policy_version 13740 (0.0009) [2023-10-12 20:31:50,863][44958] Updated weights for policy 0, policy_version 13750 (0.0007) [2023-10-12 20:31:51,232][44958] Updated weights for policy 0, policy_version 13760 (0.0009) [2023-10-12 20:31:51,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28246016. Throughput: 0: 1634.6, 1: 1651.0. Samples: 7065220. Policy #0 lag: (min: 21.0, avg: 21.5, max: 36.0) [2023-10-12 20:31:51,444][43579] Avg episode reward: [(0, '272.430'), (1, '247.380')] [2023-10-12 20:31:51,994][44959] Updated weights for policy 1, policy_version 13830 (0.0009) [2023-10-12 20:31:52,359][44959] Updated weights for policy 1, policy_version 13840 (0.0009) [2023-10-12 20:31:52,737][44959] Updated weights for policy 1, policy_version 13850 (0.0007) [2023-10-12 20:31:55,487][44958] Updated weights for policy 0, policy_version 13770 (0.0008) [2023-10-12 20:31:55,870][44958] Updated weights for policy 0, policy_version 13780 (0.0008) [2023-10-12 20:31:56,241][44958] Updated weights for policy 0, policy_version 13790 (0.0009) [2023-10-12 20:31:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28311552. Throughput: 0: 1634.0, 1: 1654.9. Samples: 7085368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:31:56,444][43579] Avg episode reward: [(0, '274.510'), (1, '252.000')] [2023-10-12 20:31:56,445][44518] Saving new best policy, reward=274.510! [2023-10-12 20:31:56,688][44959] Updated weights for policy 1, policy_version 13860 (0.0008) [2023-10-12 20:31:57,065][44959] Updated weights for policy 1, policy_version 13870 (0.0009) [2023-10-12 20:31:57,425][44959] Updated weights for policy 1, policy_version 13880 (0.0009) [2023-10-12 20:32:00,207][44958] Updated weights for policy 0, policy_version 13800 (0.0010) [2023-10-12 20:32:00,579][44958] Updated weights for policy 0, policy_version 13810 (0.0008) [2023-10-12 20:32:00,955][44958] Updated weights for policy 0, policy_version 13820 (0.0009) [2023-10-12 20:32:01,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28377088. Throughput: 0: 1632.0, 1: 1652.9. Samples: 7104812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:32:01,443][43579] Avg episode reward: [(0, '272.810'), (1, '257.750')] [2023-10-12 20:32:01,588][44959] Updated weights for policy 1, policy_version 13890 (0.0010) [2023-10-12 20:32:01,954][44959] Updated weights for policy 1, policy_version 13900 (0.0009) [2023-10-12 20:32:02,328][44959] Updated weights for policy 1, policy_version 13910 (0.0009) [2023-10-12 20:32:02,698][44959] Updated weights for policy 1, policy_version 13920 (0.0009) [2023-10-12 20:32:05,074][44958] Updated weights for policy 0, policy_version 13830 (0.0010) [2023-10-12 20:32:05,453][44958] Updated weights for policy 0, policy_version 13840 (0.0007) [2023-10-12 20:32:05,825][44958] Updated weights for policy 0, policy_version 13850 (0.0008) [2023-10-12 20:32:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28442624. Throughput: 0: 1639.6, 1: 1652.0. Samples: 7114900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:32:06,444][43579] Avg episode reward: [(0, '276.170'), (1, '259.290')] [2023-10-12 20:32:06,445][44518] Saving new best policy, reward=276.170! [2023-10-12 20:32:06,776][44959] Updated weights for policy 1, policy_version 13930 (0.0008) [2023-10-12 20:32:07,133][44959] Updated weights for policy 1, policy_version 13940 (0.0009) [2023-10-12 20:32:07,511][44959] Updated weights for policy 1, policy_version 13950 (0.0008) [2023-10-12 20:32:10,020][44958] Updated weights for policy 0, policy_version 13860 (0.0008) [2023-10-12 20:32:10,388][44958] Updated weights for policy 0, policy_version 13870 (0.0010) [2023-10-12 20:32:10,778][44958] Updated weights for policy 0, policy_version 13880 (0.0008) [2023-10-12 20:32:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28508160. Throughput: 0: 1633.3, 1: 1652.6. Samples: 7135034. Policy #0 lag: (min: 18.0, avg: 21.7, max: 50.0) [2023-10-12 20:32:11,443][43579] Avg episode reward: [(0, '275.120'), (1, '259.120')] [2023-10-12 20:32:11,633][44959] Updated weights for policy 1, policy_version 13960 (0.0009) [2023-10-12 20:32:12,001][44959] Updated weights for policy 1, policy_version 13970 (0.0009) [2023-10-12 20:32:12,373][44959] Updated weights for policy 1, policy_version 13980 (0.0008) [2023-10-12 20:32:14,717][44958] Updated weights for policy 0, policy_version 13890 (0.0008) [2023-10-12 20:32:15,078][44958] Updated weights for policy 0, policy_version 13900 (0.0009) [2023-10-12 20:32:15,461][44958] Updated weights for policy 0, policy_version 13910 (0.0010) [2023-10-12 20:32:15,830][44958] Updated weights for policy 0, policy_version 13920 (0.0008) [2023-10-12 20:32:16,260][44959] Updated weights for policy 1, policy_version 13990 (0.0008) [2023-10-12 20:32:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28573696. Throughput: 0: 1644.5, 1: 1653.5. Samples: 7154998. Policy #0 lag: (min: 18.0, avg: 21.7, max: 50.0) [2023-10-12 20:32:16,444][43579] Avg episode reward: [(0, '267.190'), (1, '255.960')] [2023-10-12 20:32:16,636][44959] Updated weights for policy 1, policy_version 14000 (0.0007) [2023-10-12 20:32:17,005][44959] Updated weights for policy 1, policy_version 14010 (0.0007) [2023-10-12 20:32:20,096][44958] Updated weights for policy 0, policy_version 13930 (0.0007) [2023-10-12 20:32:20,467][44958] Updated weights for policy 0, policy_version 13940 (0.0007) [2023-10-12 20:32:20,829][44958] Updated weights for policy 0, policy_version 13950 (0.0007) [2023-10-12 20:32:21,380][44959] Updated weights for policy 1, policy_version 14020 (0.0010) [2023-10-12 20:32:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28639232. Throughput: 0: 1652.4, 1: 1651.5. Samples: 7165036. Policy #0 lag: (min: 18.0, avg: 21.7, max: 50.0) [2023-10-12 20:32:21,444][43579] Avg episode reward: [(0, '268.160'), (1, '255.940')] [2023-10-12 20:32:21,755][44959] Updated weights for policy 1, policy_version 14030 (0.0008) [2023-10-12 20:32:22,126][44959] Updated weights for policy 1, policy_version 14040 (0.0010) [2023-10-12 20:32:25,024][44958] Updated weights for policy 0, policy_version 13960 (0.0009) [2023-10-12 20:32:25,397][44958] Updated weights for policy 0, policy_version 13970 (0.0010) [2023-10-12 20:32:25,762][44958] Updated weights for policy 0, policy_version 13980 (0.0008) [2023-10-12 20:32:26,210][44959] Updated weights for policy 1, policy_version 14050 (0.0009) [2023-10-12 20:32:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28704768. Throughput: 0: 1645.0, 1: 1652.0. Samples: 7184640. Policy #0 lag: (min: 24.0, avg: 45.3, max: 56.0) [2023-10-12 20:32:26,444][43579] Avg episode reward: [(0, '267.540'), (1, '252.150')] [2023-10-12 20:32:26,568][44959] Updated weights for policy 1, policy_version 14060 (0.0008) [2023-10-12 20:32:26,936][44959] Updated weights for policy 1, policy_version 14070 (0.0007) [2023-10-12 20:32:27,307][44959] Updated weights for policy 1, policy_version 14080 (0.0009) [2023-10-12 20:32:30,045][44958] Updated weights for policy 0, policy_version 13990 (0.0008) [2023-10-12 20:32:30,416][44958] Updated weights for policy 0, policy_version 14000 (0.0007) [2023-10-12 20:32:30,794][44958] Updated weights for policy 0, policy_version 14010 (0.0008) [2023-10-12 20:32:31,265][44959] Updated weights for policy 1, policy_version 14090 (0.0007) [2023-10-12 20:32:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 28770304. Throughput: 0: 1646.5, 1: 1654.2. Samples: 7204088. Policy #0 lag: (min: 24.0, avg: 45.3, max: 56.0) [2023-10-12 20:32:31,443][43579] Avg episode reward: [(0, '262.550'), (1, '254.860')] [2023-10-12 20:32:31,632][44959] Updated weights for policy 1, policy_version 14100 (0.0008) [2023-10-12 20:32:32,002][44959] Updated weights for policy 1, policy_version 14110 (0.0008) [2023-10-12 20:32:34,792][44958] Updated weights for policy 0, policy_version 14020 (0.0009) [2023-10-12 20:32:35,177][44958] Updated weights for policy 0, policy_version 14030 (0.0007) [2023-10-12 20:32:35,549][44958] Updated weights for policy 0, policy_version 14040 (0.0007) [2023-10-12 20:32:36,252][44959] Updated weights for policy 1, policy_version 14120 (0.0009) [2023-10-12 20:32:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28835840. Throughput: 0: 1654.3, 1: 1664.7. Samples: 7214572. Policy #0 lag: (min: 24.0, avg: 45.3, max: 56.0) [2023-10-12 20:32:36,443][43579] Avg episode reward: [(0, '262.270'), (1, '255.780')] [2023-10-12 20:32:36,636][44959] Updated weights for policy 1, policy_version 14130 (0.0009) [2023-10-12 20:32:37,006][44959] Updated weights for policy 1, policy_version 14140 (0.0008) [2023-10-12 20:32:39,708][44958] Updated weights for policy 0, policy_version 14050 (0.0007) [2023-10-12 20:32:40,085][44958] Updated weights for policy 0, policy_version 14060 (0.0008) [2023-10-12 20:32:40,452][44958] Updated weights for policy 0, policy_version 14070 (0.0008) [2023-10-12 20:32:40,828][44958] Updated weights for policy 0, policy_version 14080 (0.0009) [2023-10-12 20:32:41,138][44959] Updated weights for policy 1, policy_version 14150 (0.0010) [2023-10-12 20:32:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28901376. Throughput: 0: 1645.7, 1: 1659.5. Samples: 7234098. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-12 20:32:41,443][43579] Avg episode reward: [(0, '268.130'), (1, '255.340')] [2023-10-12 20:32:41,516][44959] Updated weights for policy 1, policy_version 14160 (0.0008) [2023-10-12 20:32:41,886][44959] Updated weights for policy 1, policy_version 14170 (0.0008) [2023-10-12 20:32:45,083][44958] Updated weights for policy 0, policy_version 14090 (0.0007) [2023-10-12 20:32:45,453][44958] Updated weights for policy 0, policy_version 14100 (0.0008) [2023-10-12 20:32:45,834][44958] Updated weights for policy 0, policy_version 14110 (0.0009) [2023-10-12 20:32:45,885][44959] Updated weights for policy 1, policy_version 14180 (0.0008) [2023-10-12 20:32:46,258][44959] Updated weights for policy 1, policy_version 14190 (0.0007) [2023-10-12 20:32:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 28966912. Throughput: 0: 1652.1, 1: 1655.7. Samples: 7253662. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-12 20:32:46,443][43579] Avg episode reward: [(0, '268.040'), (1, '253.340')] [2023-10-12 20:32:46,619][44959] Updated weights for policy 1, policy_version 14200 (0.0008) [2023-10-12 20:32:49,887][44958] Updated weights for policy 0, policy_version 14120 (0.0009) [2023-10-12 20:32:50,256][44958] Updated weights for policy 0, policy_version 14130 (0.0007) [2023-10-12 20:32:50,623][44958] Updated weights for policy 0, policy_version 14140 (0.0008) [2023-10-12 20:32:50,762][44959] Updated weights for policy 1, policy_version 14210 (0.0008) [2023-10-12 20:32:51,121][44959] Updated weights for policy 1, policy_version 14220 (0.0008) [2023-10-12 20:32:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 29032448. Throughput: 0: 1651.8, 1: 1663.0. Samples: 7264064. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-12 20:32:51,443][43579] Avg episode reward: [(0, '268.640'), (1, '253.210')] [2023-10-12 20:32:51,489][44959] Updated weights for policy 1, policy_version 14230 (0.0009) [2023-10-12 20:32:51,861][44959] Updated weights for policy 1, policy_version 14240 (0.0011) [2023-10-12 20:32:54,901][44958] Updated weights for policy 0, policy_version 14150 (0.0009) [2023-10-12 20:32:55,274][44958] Updated weights for policy 0, policy_version 14160 (0.0010) [2023-10-12 20:32:55,646][44958] Updated weights for policy 0, policy_version 14170 (0.0008) [2023-10-12 20:32:56,117][44959] Updated weights for policy 1, policy_version 14250 (0.0007) [2023-10-12 20:32:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29097984. Throughput: 0: 1646.2, 1: 1662.4. Samples: 7283922. Policy #0 lag: (min: 30.0, avg: 30.0, max: 33.0) [2023-10-12 20:32:56,444][43579] Avg episode reward: [(0, '268.210'), (1, '254.180')] [2023-10-12 20:32:56,485][44959] Updated weights for policy 1, policy_version 14260 (0.0008) [2023-10-12 20:32:56,850][44959] Updated weights for policy 1, policy_version 14270 (0.0008) [2023-10-12 20:32:59,715][44958] Updated weights for policy 0, policy_version 14180 (0.0008) [2023-10-12 20:33:00,082][44958] Updated weights for policy 0, policy_version 14190 (0.0009) [2023-10-12 20:33:00,460][44958] Updated weights for policy 0, policy_version 14200 (0.0007) [2023-10-12 20:33:00,963][44959] Updated weights for policy 1, policy_version 14280 (0.0008) [2023-10-12 20:33:01,333][44959] Updated weights for policy 1, policy_version 14290 (0.0009) [2023-10-12 20:33:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29163520. Throughput: 0: 1645.2, 1: 1651.3. Samples: 7303340. Policy #0 lag: (min: 30.0, avg: 30.0, max: 33.0) [2023-10-12 20:33:01,444][43579] Avg episode reward: [(0, '267.980'), (1, '254.230')] [2023-10-12 20:33:01,710][44959] Updated weights for policy 1, policy_version 14300 (0.0009) [2023-10-12 20:33:04,492][44958] Updated weights for policy 0, policy_version 14210 (0.0007) [2023-10-12 20:33:04,861][44958] Updated weights for policy 0, policy_version 14220 (0.0007) [2023-10-12 20:33:05,234][44958] Updated weights for policy 0, policy_version 14230 (0.0009) [2023-10-12 20:33:05,614][44958] Updated weights for policy 0, policy_version 14240 (0.0010) [2023-10-12 20:33:05,947][44959] Updated weights for policy 1, policy_version 14310 (0.0007) [2023-10-12 20:33:06,313][44959] Updated weights for policy 1, policy_version 14320 (0.0010) [2023-10-12 20:33:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 29229056. Throughput: 0: 1648.3, 1: 1661.2. Samples: 7313962. Policy #0 lag: (min: 30.0, avg: 30.0, max: 33.0) [2023-10-12 20:33:06,443][43579] Avg episode reward: [(0, '269.730'), (1, '254.580')] [2023-10-12 20:33:06,685][44959] Updated weights for policy 1, policy_version 14330 (0.0008) [2023-10-12 20:33:09,834][44958] Updated weights for policy 0, policy_version 14250 (0.0009) [2023-10-12 20:33:10,211][44958] Updated weights for policy 0, policy_version 14260 (0.0007) [2023-10-12 20:33:10,594][44958] Updated weights for policy 0, policy_version 14270 (0.0008) [2023-10-12 20:33:10,798][44959] Updated weights for policy 1, policy_version 14340 (0.0008) [2023-10-12 20:33:11,158][44959] Updated weights for policy 1, policy_version 14350 (0.0007) [2023-10-12 20:33:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29294592. Throughput: 0: 1648.1, 1: 1660.9. Samples: 7333542. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-12 20:33:11,443][43579] Avg episode reward: [(0, '273.240'), (1, '260.750')] [2023-10-12 20:33:11,532][44959] Updated weights for policy 1, policy_version 14360 (0.0008) [2023-10-12 20:33:14,684][44958] Updated weights for policy 0, policy_version 14280 (0.0008) [2023-10-12 20:33:15,054][44958] Updated weights for policy 0, policy_version 14290 (0.0008) [2023-10-12 20:33:15,426][44958] Updated weights for policy 0, policy_version 14300 (0.0008) [2023-10-12 20:33:15,554][44959] Updated weights for policy 1, policy_version 14370 (0.0007) [2023-10-12 20:33:15,936][44959] Updated weights for policy 1, policy_version 14380 (0.0011) [2023-10-12 20:33:16,306][44959] Updated weights for policy 1, policy_version 14390 (0.0011) [2023-10-12 20:33:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29360128. Throughput: 0: 1654.1, 1: 1649.4. Samples: 7352746. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-12 20:33:16,444][43579] Avg episode reward: [(0, '267.470'), (1, '262.820')] [2023-10-12 20:33:16,688][44959] Updated weights for policy 1, policy_version 14400 (0.0009) [2023-10-12 20:33:19,561][44958] Updated weights for policy 0, policy_version 14310 (0.0009) [2023-10-12 20:33:19,925][44958] Updated weights for policy 0, policy_version 14320 (0.0010) [2023-10-12 20:33:20,310][44958] Updated weights for policy 0, policy_version 14330 (0.0009) [2023-10-12 20:33:20,839][44959] Updated weights for policy 1, policy_version 14410 (0.0007) [2023-10-12 20:33:21,213][44959] Updated weights for policy 1, policy_version 14420 (0.0008) [2023-10-12 20:33:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29425664. Throughput: 0: 1652.3, 1: 1660.6. Samples: 7363652. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-12 20:33:21,444][43579] Avg episode reward: [(0, '268.080'), (1, '262.450')] [2023-10-12 20:33:21,566][44959] Updated weights for policy 1, policy_version 14430 (0.0008) [2023-10-12 20:33:24,563][44958] Updated weights for policy 0, policy_version 14340 (0.0008) [2023-10-12 20:33:24,948][44958] Updated weights for policy 0, policy_version 14350 (0.0008) [2023-10-12 20:33:25,328][44958] Updated weights for policy 0, policy_version 14360 (0.0008) [2023-10-12 20:33:25,967][44959] Updated weights for policy 1, policy_version 14440 (0.0008) [2023-10-12 20:33:26,337][44959] Updated weights for policy 1, policy_version 14450 (0.0007) [2023-10-12 20:33:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29491200. Throughput: 0: 1650.9, 1: 1655.2. Samples: 7382874. Policy #0 lag: (min: 27.0, avg: 27.1, max: 32.0) [2023-10-12 20:33:26,443][43579] Avg episode reward: [(0, '265.710'), (1, '262.780')] [2023-10-12 20:33:26,703][44959] Updated weights for policy 1, policy_version 14460 (0.0008) [2023-10-12 20:33:29,440][44958] Updated weights for policy 0, policy_version 14370 (0.0007) [2023-10-12 20:33:29,812][44958] Updated weights for policy 0, policy_version 14380 (0.0008) [2023-10-12 20:33:30,179][44958] Updated weights for policy 0, policy_version 14390 (0.0009) [2023-10-12 20:33:30,554][44958] Updated weights for policy 0, policy_version 14400 (0.0010) [2023-10-12 20:33:30,871][44959] Updated weights for policy 1, policy_version 14470 (0.0008) [2023-10-12 20:33:31,238][44959] Updated weights for policy 1, policy_version 14480 (0.0008) [2023-10-12 20:33:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29556736. Throughput: 0: 1655.6, 1: 1648.7. Samples: 7402356. Policy #0 lag: (min: 27.0, avg: 27.1, max: 32.0) [2023-10-12 20:33:31,443][43579] Avg episode reward: [(0, '266.680'), (1, '259.640')] [2023-10-12 20:33:31,597][44959] Updated weights for policy 1, policy_version 14490 (0.0009) [2023-10-12 20:33:34,646][44958] Updated weights for policy 0, policy_version 14410 (0.0010) [2023-10-12 20:33:35,011][44958] Updated weights for policy 0, policy_version 14420 (0.0008) [2023-10-12 20:33:35,389][44958] Updated weights for policy 0, policy_version 14430 (0.0008) [2023-10-12 20:33:35,591][44959] Updated weights for policy 1, policy_version 14500 (0.0009) [2023-10-12 20:33:35,963][44959] Updated weights for policy 1, policy_version 14510 (0.0008) [2023-10-12 20:33:36,331][44959] Updated weights for policy 1, policy_version 14520 (0.0009) [2023-10-12 20:33:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29622272. Throughput: 0: 1659.3, 1: 1651.5. Samples: 7413054. Policy #0 lag: (min: 27.0, avg: 27.1, max: 32.0) [2023-10-12 20:33:36,444][43579] Avg episode reward: [(0, '265.960'), (1, '251.840')] [2023-10-12 20:33:39,284][44958] Updated weights for policy 0, policy_version 14440 (0.0009) [2023-10-12 20:33:39,657][44958] Updated weights for policy 0, policy_version 14450 (0.0008) [2023-10-12 20:33:40,033][44958] Updated weights for policy 0, policy_version 14460 (0.0009) [2023-10-12 20:33:40,438][44959] Updated weights for policy 1, policy_version 14530 (0.0008) [2023-10-12 20:33:40,817][44959] Updated weights for policy 1, policy_version 14540 (0.0010) [2023-10-12 20:33:41,183][44959] Updated weights for policy 1, policy_version 14550 (0.0007) [2023-10-12 20:33:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29687808. Throughput: 0: 1649.7, 1: 1656.2. Samples: 7432690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:33:41,443][43579] Avg episode reward: [(0, '263.870'), (1, '247.260')] [2023-10-12 20:33:41,551][44959] Updated weights for policy 1, policy_version 14560 (0.0010) [2023-10-12 20:33:44,308][44958] Updated weights for policy 0, policy_version 14470 (0.0009) [2023-10-12 20:33:44,685][44958] Updated weights for policy 0, policy_version 14480 (0.0007) [2023-10-12 20:33:45,064][44958] Updated weights for policy 0, policy_version 14490 (0.0008) [2023-10-12 20:33:45,743][44959] Updated weights for policy 1, policy_version 14570 (0.0007) [2023-10-12 20:33:46,108][44959] Updated weights for policy 1, policy_version 14580 (0.0008) [2023-10-12 20:33:46,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29753344. Throughput: 0: 1663.2, 1: 1646.6. Samples: 7452280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:33:46,443][43579] Avg episode reward: [(0, '262.970'), (1, '249.690')] [2023-10-12 20:33:46,450][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000014496_14843904.pth... [2023-10-12 20:33:46,470][44959] Updated weights for policy 1, policy_version 14590 (0.0010) [2023-10-12 20:33:46,489][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000012960_13271040.pth [2023-10-12 20:33:46,544][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000014592_14942208.pth... [2023-10-12 20:33:46,574][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000013024_13336576.pth [2023-10-12 20:33:49,076][44958] Updated weights for policy 0, policy_version 14500 (0.0008) [2023-10-12 20:33:49,446][44958] Updated weights for policy 0, policy_version 14510 (0.0009) [2023-10-12 20:33:49,827][44958] Updated weights for policy 0, policy_version 14520 (0.0009) [2023-10-12 20:33:50,679][44959] Updated weights for policy 1, policy_version 14600 (0.0009) [2023-10-12 20:33:51,050][44959] Updated weights for policy 1, policy_version 14610 (0.0008) [2023-10-12 20:33:51,418][44959] Updated weights for policy 1, policy_version 14620 (0.0010) [2023-10-12 20:33:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29818880. Throughput: 0: 1656.3, 1: 1648.1. Samples: 7462658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:33:51,443][43579] Avg episode reward: [(0, '259.920'), (1, '255.550')] [2023-10-12 20:33:54,051][44958] Updated weights for policy 0, policy_version 14530 (0.0008) [2023-10-12 20:33:54,438][44958] Updated weights for policy 0, policy_version 14540 (0.0009) [2023-10-12 20:33:54,811][44958] Updated weights for policy 0, policy_version 14550 (0.0009) [2023-10-12 20:33:55,183][44958] Updated weights for policy 0, policy_version 14560 (0.0008) [2023-10-12 20:33:55,556][44959] Updated weights for policy 1, policy_version 14630 (0.0009) [2023-10-12 20:33:55,939][44959] Updated weights for policy 1, policy_version 14640 (0.0010) [2023-10-12 20:33:56,303][44959] Updated weights for policy 1, policy_version 14650 (0.0008) [2023-10-12 20:33:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 29884416. Throughput: 0: 1645.6, 1: 1653.4. Samples: 7481998. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 20:33:56,443][43579] Avg episode reward: [(0, '263.960'), (1, '255.720')] [2023-10-12 20:33:59,426][44958] Updated weights for policy 0, policy_version 14570 (0.0008) [2023-10-12 20:33:59,791][44958] Updated weights for policy 0, policy_version 14580 (0.0008) [2023-10-12 20:34:00,173][44958] Updated weights for policy 0, policy_version 14590 (0.0010) [2023-10-12 20:34:00,341][44959] Updated weights for policy 1, policy_version 14660 (0.0009) [2023-10-12 20:34:00,715][44959] Updated weights for policy 1, policy_version 14670 (0.0011) [2023-10-12 20:34:01,079][44959] Updated weights for policy 1, policy_version 14680 (0.0010) [2023-10-12 20:34:01,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 29982720. Throughput: 0: 1658.2, 1: 1645.8. Samples: 7501426. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 20:34:01,444][43579] Avg episode reward: [(0, '258.890'), (1, '254.250')] [2023-10-12 20:34:04,211][44958] Updated weights for policy 0, policy_version 14600 (0.0008) [2023-10-12 20:34:04,576][44958] Updated weights for policy 0, policy_version 14610 (0.0008) [2023-10-12 20:34:04,944][44958] Updated weights for policy 0, policy_version 14620 (0.0008) [2023-10-12 20:34:05,228][44959] Updated weights for policy 1, policy_version 14690 (0.0007) [2023-10-12 20:34:05,588][44959] Updated weights for policy 1, policy_version 14700 (0.0008) [2023-10-12 20:34:05,954][44959] Updated weights for policy 1, policy_version 14710 (0.0008) [2023-10-12 20:34:06,326][44959] Updated weights for policy 1, policy_version 14720 (0.0007) [2023-10-12 20:34:06,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 30048256. Throughput: 0: 1652.2, 1: 1647.2. Samples: 7512126. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 20:34:06,444][43579] Avg episode reward: [(0, '253.690'), (1, '250.260')] [2023-10-12 20:34:09,271][44958] Updated weights for policy 0, policy_version 14630 (0.0008) [2023-10-12 20:34:09,644][44958] Updated weights for policy 0, policy_version 14640 (0.0007) [2023-10-12 20:34:10,011][44958] Updated weights for policy 0, policy_version 14650 (0.0008) [2023-10-12 20:34:10,504][44959] Updated weights for policy 1, policy_version 14730 (0.0007) [2023-10-12 20:34:10,879][44959] Updated weights for policy 1, policy_version 14740 (0.0007) [2023-10-12 20:34:11,248][44959] Updated weights for policy 1, policy_version 14750 (0.0010) [2023-10-12 20:34:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 30113792. Throughput: 0: 1647.3, 1: 1658.9. Samples: 7531654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:34:11,443][43579] Avg episode reward: [(0, '258.770'), (1, '249.850')] [2023-10-12 20:34:13,898][44958] Updated weights for policy 0, policy_version 14660 (0.0007) [2023-10-12 20:34:14,271][44958] Updated weights for policy 0, policy_version 14670 (0.0009) [2023-10-12 20:34:14,649][44958] Updated weights for policy 0, policy_version 14680 (0.0008) [2023-10-12 20:34:15,488][44959] Updated weights for policy 1, policy_version 14760 (0.0008) [2023-10-12 20:34:15,860][44959] Updated weights for policy 1, policy_version 14770 (0.0007) [2023-10-12 20:34:16,244][44959] Updated weights for policy 1, policy_version 14780 (0.0011) [2023-10-12 20:34:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 30179328. Throughput: 0: 1661.5, 1: 1639.2. Samples: 7550888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:34:16,444][43579] Avg episode reward: [(0, '260.170'), (1, '250.540')] [2023-10-12 20:34:18,829][44958] Updated weights for policy 0, policy_version 14690 (0.0008) [2023-10-12 20:34:19,199][44958] Updated weights for policy 0, policy_version 14700 (0.0010) [2023-10-12 20:34:19,569][44958] Updated weights for policy 0, policy_version 14710 (0.0011) [2023-10-12 20:34:19,940][44958] Updated weights for policy 0, policy_version 14720 (0.0009) [2023-10-12 20:34:20,320][44959] Updated weights for policy 1, policy_version 14790 (0.0009) [2023-10-12 20:34:20,690][44959] Updated weights for policy 1, policy_version 14800 (0.0010) [2023-10-12 20:34:21,068][44959] Updated weights for policy 1, policy_version 14810 (0.0007) [2023-10-12 20:34:21,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 30244864. Throughput: 0: 1644.6, 1: 1655.4. Samples: 7561554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:34:21,444][43579] Avg episode reward: [(0, '254.720'), (1, '240.310')] [2023-10-12 20:34:24,155][44958] Updated weights for policy 0, policy_version 14730 (0.0008) [2023-10-12 20:34:24,534][44958] Updated weights for policy 0, policy_version 14740 (0.0007) [2023-10-12 20:34:24,897][44958] Updated weights for policy 0, policy_version 14750 (0.0008) [2023-10-12 20:34:25,109][44959] Updated weights for policy 1, policy_version 14820 (0.0009) [2023-10-12 20:34:25,480][44959] Updated weights for policy 1, policy_version 14830 (0.0009) [2023-10-12 20:34:25,847][44959] Updated weights for policy 1, policy_version 14840 (0.0009) [2023-10-12 20:34:26,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 30310400. Throughput: 0: 1647.5, 1: 1643.8. Samples: 7580798. Policy #0 lag: (min: 5.0, avg: 15.5, max: 37.0) [2023-10-12 20:34:26,443][43579] Avg episode reward: [(0, '263.970'), (1, '246.440')] [2023-10-12 20:34:28,911][44958] Updated weights for policy 0, policy_version 14760 (0.0010) [2023-10-12 20:34:29,291][44958] Updated weights for policy 0, policy_version 14770 (0.0008) [2023-10-12 20:34:29,656][44958] Updated weights for policy 0, policy_version 14780 (0.0008) [2023-10-12 20:34:30,184][44959] Updated weights for policy 1, policy_version 14850 (0.0009) [2023-10-12 20:34:30,548][44959] Updated weights for policy 1, policy_version 14860 (0.0008) [2023-10-12 20:34:30,916][44959] Updated weights for policy 1, policy_version 14870 (0.0007) [2023-10-12 20:34:31,292][44959] Updated weights for policy 1, policy_version 14880 (0.0009) [2023-10-12 20:34:31,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 30375936. Throughput: 0: 1657.8, 1: 1637.9. Samples: 7600586. Policy #0 lag: (min: 5.0, avg: 15.5, max: 37.0) [2023-10-12 20:34:31,444][43579] Avg episode reward: [(0, '259.160'), (1, '244.580')] [2023-10-12 20:34:33,651][44958] Updated weights for policy 0, policy_version 14790 (0.0011) [2023-10-12 20:34:34,035][44958] Updated weights for policy 0, policy_version 14800 (0.0010) [2023-10-12 20:34:34,405][44958] Updated weights for policy 0, policy_version 14810 (0.0010) [2023-10-12 20:34:35,485][44959] Updated weights for policy 1, policy_version 14890 (0.0008) [2023-10-12 20:34:35,856][44959] Updated weights for policy 1, policy_version 14900 (0.0008) [2023-10-12 20:34:36,218][44959] Updated weights for policy 1, policy_version 14910 (0.0008) [2023-10-12 20:34:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 30441472. Throughput: 0: 1648.6, 1: 1648.6. Samples: 7611032. Policy #0 lag: (min: 5.0, avg: 15.5, max: 37.0) [2023-10-12 20:34:36,444][43579] Avg episode reward: [(0, '264.550'), (1, '247.540')] [2023-10-12 20:34:38,624][44958] Updated weights for policy 0, policy_version 14820 (0.0010) [2023-10-12 20:34:39,000][44958] Updated weights for policy 0, policy_version 14830 (0.0009) [2023-10-12 20:34:39,377][44958] Updated weights for policy 0, policy_version 14840 (0.0007) [2023-10-12 20:34:40,360][44959] Updated weights for policy 1, policy_version 14920 (0.0007) [2023-10-12 20:34:40,733][44959] Updated weights for policy 1, policy_version 14930 (0.0008) [2023-10-12 20:34:41,090][44959] Updated weights for policy 1, policy_version 14940 (0.0009) [2023-10-12 20:34:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 30507008. Throughput: 0: 1662.7, 1: 1645.8. Samples: 7630878. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 20:34:41,444][43579] Avg episode reward: [(0, '267.950'), (1, '252.040')] [2023-10-12 20:34:43,340][44958] Updated weights for policy 0, policy_version 14850 (0.0010) [2023-10-12 20:34:43,709][44958] Updated weights for policy 0, policy_version 14860 (0.0009) [2023-10-12 20:34:44,072][44958] Updated weights for policy 0, policy_version 14870 (0.0007) [2023-10-12 20:34:44,451][44958] Updated weights for policy 0, policy_version 14880 (0.0008) [2023-10-12 20:34:45,128][44959] Updated weights for policy 1, policy_version 14950 (0.0008) [2023-10-12 20:34:45,505][44959] Updated weights for policy 1, policy_version 14960 (0.0007) [2023-10-12 20:34:45,873][44959] Updated weights for policy 1, policy_version 14970 (0.0008) [2023-10-12 20:34:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 30572544. Throughput: 0: 1669.7, 1: 1641.0. Samples: 7650410. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 20:34:46,443][43579] Avg episode reward: [(0, '271.910'), (1, '255.000')] [2023-10-12 20:34:48,573][44958] Updated weights for policy 0, policy_version 14890 (0.0007) [2023-10-12 20:34:48,937][44958] Updated weights for policy 0, policy_version 14900 (0.0009) [2023-10-12 20:34:49,315][44958] Updated weights for policy 0, policy_version 14910 (0.0008) [2023-10-12 20:34:50,189][44959] Updated weights for policy 1, policy_version 14980 (0.0009) [2023-10-12 20:34:50,546][44959] Updated weights for policy 1, policy_version 14990 (0.0011) [2023-10-12 20:34:50,920][44959] Updated weights for policy 1, policy_version 15000 (0.0010) [2023-10-12 20:34:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 30638080. Throughput: 0: 1650.8, 1: 1646.8. Samples: 7660516. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 20:34:51,444][43579] Avg episode reward: [(0, '273.880'), (1, '259.470')] [2023-10-12 20:34:53,408][44958] Updated weights for policy 0, policy_version 14920 (0.0008) [2023-10-12 20:34:53,791][44958] Updated weights for policy 0, policy_version 14930 (0.0007) [2023-10-12 20:34:54,162][44958] Updated weights for policy 0, policy_version 14940 (0.0007) [2023-10-12 20:34:55,135][44959] Updated weights for policy 1, policy_version 15010 (0.0010) [2023-10-12 20:34:55,559][44959] Updated weights for policy 1, policy_version 15020 (0.0010) [2023-10-12 20:34:55,937][44959] Updated weights for policy 1, policy_version 15030 (0.0008) [2023-10-12 20:34:56,302][44959] Updated weights for policy 1, policy_version 15040 (0.0010) [2023-10-12 20:34:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 30703616. Throughput: 0: 1666.9, 1: 1642.3. Samples: 7680566. Policy #0 lag: (min: 5.0, avg: 5.4, max: 18.0) [2023-10-12 20:34:56,443][43579] Avg episode reward: [(0, '262.120'), (1, '254.810')] [2023-10-12 20:34:58,440][44958] Updated weights for policy 0, policy_version 14950 (0.0008) [2023-10-12 20:34:58,817][44958] Updated weights for policy 0, policy_version 14960 (0.0011) [2023-10-12 20:34:59,182][44958] Updated weights for policy 0, policy_version 14970 (0.0007) [2023-10-12 20:35:00,418][44959] Updated weights for policy 1, policy_version 15050 (0.0010) [2023-10-12 20:35:00,796][44959] Updated weights for policy 1, policy_version 15060 (0.0010) [2023-10-12 20:35:01,164][44959] Updated weights for policy 1, policy_version 15070 (0.0010) [2023-10-12 20:35:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30769152. Throughput: 0: 1661.1, 1: 1640.5. Samples: 7699462. Policy #0 lag: (min: 5.0, avg: 5.4, max: 18.0) [2023-10-12 20:35:01,444][43579] Avg episode reward: [(0, '261.510'), (1, '261.640')] [2023-10-12 20:35:03,439][44958] Updated weights for policy 0, policy_version 14980 (0.0008) [2023-10-12 20:35:03,825][44958] Updated weights for policy 0, policy_version 14990 (0.0010) [2023-10-12 20:35:04,187][44958] Updated weights for policy 0, policy_version 15000 (0.0010) [2023-10-12 20:35:05,371][44959] Updated weights for policy 1, policy_version 15080 (0.0011) [2023-10-12 20:35:05,741][44959] Updated weights for policy 1, policy_version 15090 (0.0009) [2023-10-12 20:35:06,125][44959] Updated weights for policy 1, policy_version 15100 (0.0009) [2023-10-12 20:35:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30834688. Throughput: 0: 1648.9, 1: 1638.1. Samples: 7709468. Policy #0 lag: (min: 5.0, avg: 5.4, max: 18.0) [2023-10-12 20:35:06,443][43579] Avg episode reward: [(0, '262.530'), (1, '256.220')] [2023-10-12 20:35:08,453][44958] Updated weights for policy 0, policy_version 15010 (0.0009) [2023-10-12 20:35:08,822][44958] Updated weights for policy 0, policy_version 15020 (0.0009) [2023-10-12 20:35:09,189][44958] Updated weights for policy 0, policy_version 15030 (0.0009) [2023-10-12 20:35:09,559][44958] Updated weights for policy 0, policy_version 15040 (0.0009) [2023-10-12 20:35:10,312][44959] Updated weights for policy 1, policy_version 15110 (0.0010) [2023-10-12 20:35:10,679][44959] Updated weights for policy 1, policy_version 15120 (0.0009) [2023-10-12 20:35:11,047][44959] Updated weights for policy 1, policy_version 15130 (0.0010) [2023-10-12 20:35:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 30900224. Throughput: 0: 1653.4, 1: 1639.4. Samples: 7728976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:35:11,443][43579] Avg episode reward: [(0, '263.000'), (1, '254.450')] [2023-10-12 20:35:13,862][44958] Updated weights for policy 0, policy_version 15050 (0.0008) [2023-10-12 20:35:14,246][44958] Updated weights for policy 0, policy_version 15060 (0.0009) [2023-10-12 20:35:14,619][44958] Updated weights for policy 0, policy_version 15070 (0.0008) [2023-10-12 20:35:15,170][44959] Updated weights for policy 1, policy_version 15140 (0.0009) [2023-10-12 20:35:15,542][44959] Updated weights for policy 1, policy_version 15150 (0.0007) [2023-10-12 20:35:15,913][44959] Updated weights for policy 1, policy_version 15160 (0.0009) [2023-10-12 20:35:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 30965760. Throughput: 0: 1641.7, 1: 1639.5. Samples: 7748242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:35:16,443][43579] Avg episode reward: [(0, '259.430'), (1, '250.900')] [2023-10-12 20:35:19,007][44958] Updated weights for policy 0, policy_version 15080 (0.0010) [2023-10-12 20:35:19,372][44958] Updated weights for policy 0, policy_version 15090 (0.0007) [2023-10-12 20:35:19,745][44958] Updated weights for policy 0, policy_version 15100 (0.0008) [2023-10-12 20:35:20,162][44959] Updated weights for policy 1, policy_version 15170 (0.0008) [2023-10-12 20:35:20,530][44959] Updated weights for policy 1, policy_version 15180 (0.0007) [2023-10-12 20:35:20,894][44959] Updated weights for policy 1, policy_version 15190 (0.0008) [2023-10-12 20:35:21,265][44959] Updated weights for policy 1, policy_version 15200 (0.0010) [2023-10-12 20:35:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 31031296. Throughput: 0: 1639.6, 1: 1642.6. Samples: 7758730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:35:21,443][43579] Avg episode reward: [(0, '266.340'), (1, '257.900')] [2023-10-12 20:35:23,762][44958] Updated weights for policy 0, policy_version 15110 (0.0010) [2023-10-12 20:35:24,124][44958] Updated weights for policy 0, policy_version 15120 (0.0008) [2023-10-12 20:35:24,501][44958] Updated weights for policy 0, policy_version 15130 (0.0008) [2023-10-12 20:35:25,263][44959] Updated weights for policy 1, policy_version 15210 (0.0007) [2023-10-12 20:35:25,637][44959] Updated weights for policy 1, policy_version 15220 (0.0007) [2023-10-12 20:35:26,006][44959] Updated weights for policy 1, policy_version 15230 (0.0009) [2023-10-12 20:35:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31096832. Throughput: 0: 1630.4, 1: 1639.8. Samples: 7778036. Policy #0 lag: (min: 24.0, avg: 53.2, max: 56.0) [2023-10-12 20:35:26,443][43579] Avg episode reward: [(0, '269.840'), (1, '251.450')] [2023-10-12 20:35:28,595][44958] Updated weights for policy 0, policy_version 15140 (0.0009) [2023-10-12 20:35:28,956][44958] Updated weights for policy 0, policy_version 15150 (0.0010) [2023-10-12 20:35:29,329][44958] Updated weights for policy 0, policy_version 15160 (0.0011) [2023-10-12 20:35:30,241][44959] Updated weights for policy 1, policy_version 15240 (0.0010) [2023-10-12 20:35:30,604][44959] Updated weights for policy 1, policy_version 15250 (0.0010) [2023-10-12 20:35:30,973][44959] Updated weights for policy 1, policy_version 15260 (0.0008) [2023-10-12 20:35:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31162368. Throughput: 0: 1622.4, 1: 1645.5. Samples: 7797468. Policy #0 lag: (min: 24.0, avg: 53.2, max: 56.0) [2023-10-12 20:35:31,443][43579] Avg episode reward: [(0, '271.060'), (1, '254.680')] [2023-10-12 20:35:33,552][44958] Updated weights for policy 0, policy_version 15170 (0.0008) [2023-10-12 20:35:33,919][44958] Updated weights for policy 0, policy_version 15180 (0.0010) [2023-10-12 20:35:34,298][44958] Updated weights for policy 0, policy_version 15190 (0.0011) [2023-10-12 20:35:34,664][44958] Updated weights for policy 0, policy_version 15200 (0.0010) [2023-10-12 20:35:35,051][44959] Updated weights for policy 1, policy_version 15270 (0.0009) [2023-10-12 20:35:35,409][44959] Updated weights for policy 1, policy_version 15280 (0.0011) [2023-10-12 20:35:35,784][44959] Updated weights for policy 1, policy_version 15290 (0.0009) [2023-10-12 20:35:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 31227904. Throughput: 0: 1627.7, 1: 1648.2. Samples: 7807930. Policy #0 lag: (min: 24.0, avg: 53.2, max: 56.0) [2023-10-12 20:35:36,443][43579] Avg episode reward: [(0, '262.450'), (1, '258.550')] [2023-10-12 20:35:38,919][44958] Updated weights for policy 0, policy_version 15210 (0.0011) [2023-10-12 20:35:39,286][44958] Updated weights for policy 0, policy_version 15220 (0.0011) [2023-10-12 20:35:39,656][44958] Updated weights for policy 0, policy_version 15230 (0.0010) [2023-10-12 20:35:40,112][44959] Updated weights for policy 1, policy_version 15300 (0.0008) [2023-10-12 20:35:40,515][44959] Updated weights for policy 1, policy_version 15310 (0.0009) [2023-10-12 20:35:40,887][44959] Updated weights for policy 1, policy_version 15320 (0.0010) [2023-10-12 20:35:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31293440. Throughput: 0: 1615.6, 1: 1647.4. Samples: 7827402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:35:41,443][43579] Avg episode reward: [(0, '258.110'), (1, '249.720')] [2023-10-12 20:35:43,844][44958] Updated weights for policy 0, policy_version 15240 (0.0007) [2023-10-12 20:35:44,226][44958] Updated weights for policy 0, policy_version 15250 (0.0008) [2023-10-12 20:35:44,597][44958] Updated weights for policy 0, policy_version 15260 (0.0008) [2023-10-12 20:35:44,798][44959] Updated weights for policy 1, policy_version 15330 (0.0009) [2023-10-12 20:35:45,165][44959] Updated weights for policy 1, policy_version 15340 (0.0008) [2023-10-12 20:35:45,534][44959] Updated weights for policy 1, policy_version 15350 (0.0007) [2023-10-12 20:35:45,897][44959] Updated weights for policy 1, policy_version 15360 (0.0007) [2023-10-12 20:35:46,443][43579] Fps is (10 sec: 13106.4, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 31358976. Throughput: 0: 1624.6, 1: 1653.1. Samples: 7846960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:35:46,444][43579] Avg episode reward: [(0, '261.810'), (1, '252.110')] [2023-10-12 20:35:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000015360_15728640.pth... [2023-10-12 20:35:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000015264_15630336.pth... [2023-10-12 20:35:46,484][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000013824_14155776.pth [2023-10-12 20:35:46,488][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000015360_15728640.pth [2023-10-12 20:35:46,488][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000013728_14057472.pth [2023-10-12 20:35:46,492][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000015264_15630336.pth [2023-10-12 20:35:48,852][44958] Updated weights for policy 0, policy_version 15270 (0.0008) [2023-10-12 20:35:49,236][44958] Updated weights for policy 0, policy_version 15280 (0.0010) [2023-10-12 20:35:49,621][44958] Updated weights for policy 0, policy_version 15290 (0.0008) [2023-10-12 20:35:49,829][44959] Updated weights for policy 1, policy_version 15370 (0.0008) [2023-10-12 20:35:50,201][44959] Updated weights for policy 1, policy_version 15380 (0.0007) [2023-10-12 20:35:50,567][44959] Updated weights for policy 1, policy_version 15390 (0.0008) [2023-10-12 20:35:51,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 31424512. Throughput: 0: 1632.3, 1: 1660.1. Samples: 7857628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:35:51,444][43579] Avg episode reward: [(0, '263.730'), (1, '254.550')] [2023-10-12 20:35:53,722][44958] Updated weights for policy 0, policy_version 15300 (0.0008) [2023-10-12 20:35:54,086][44958] Updated weights for policy 0, policy_version 15310 (0.0009) [2023-10-12 20:35:54,467][44958] Updated weights for policy 0, policy_version 15320 (0.0009) [2023-10-12 20:35:54,845][44959] Updated weights for policy 1, policy_version 15400 (0.0007) [2023-10-12 20:35:55,215][44959] Updated weights for policy 1, policy_version 15410 (0.0007) [2023-10-12 20:35:55,581][44959] Updated weights for policy 1, policy_version 15420 (0.0010) [2023-10-12 20:35:56,442][43579] Fps is (10 sec: 13107.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31490048. Throughput: 0: 1631.5, 1: 1652.9. Samples: 7876774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:35:56,443][43579] Avg episode reward: [(0, '262.840'), (1, '261.310')] [2023-10-12 20:35:58,517][44958] Updated weights for policy 0, policy_version 15330 (0.0009) [2023-10-12 20:35:58,898][44958] Updated weights for policy 0, policy_version 15340 (0.0008) [2023-10-12 20:35:59,270][44958] Updated weights for policy 0, policy_version 15350 (0.0008) [2023-10-12 20:35:59,639][44958] Updated weights for policy 0, policy_version 15360 (0.0009) [2023-10-12 20:35:59,764][44959] Updated weights for policy 1, policy_version 15430 (0.0009) [2023-10-12 20:36:00,131][44959] Updated weights for policy 1, policy_version 15440 (0.0009) [2023-10-12 20:36:00,504][44959] Updated weights for policy 1, policy_version 15450 (0.0008) [2023-10-12 20:36:01,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31555584. Throughput: 0: 1631.3, 1: 1659.7. Samples: 7896338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:36:01,443][43579] Avg episode reward: [(0, '259.040'), (1, '254.370')] [2023-10-12 20:36:04,042][44958] Updated weights for policy 0, policy_version 15370 (0.0007) [2023-10-12 20:36:04,415][44958] Updated weights for policy 0, policy_version 15380 (0.0009) [2023-10-12 20:36:04,607][44959] Updated weights for policy 1, policy_version 15460 (0.0007) [2023-10-12 20:36:04,791][44958] Updated weights for policy 0, policy_version 15390 (0.0009) [2023-10-12 20:36:04,975][44959] Updated weights for policy 1, policy_version 15470 (0.0007) [2023-10-12 20:36:05,343][44959] Updated weights for policy 1, policy_version 15480 (0.0007) [2023-10-12 20:36:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31621120. Throughput: 0: 1634.2, 1: 1664.6. Samples: 7907178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:36:06,444][43579] Avg episode reward: [(0, '265.530'), (1, '259.850')] [2023-10-12 20:36:09,098][44958] Updated weights for policy 0, policy_version 15400 (0.0010) [2023-10-12 20:36:09,298][44959] Updated weights for policy 1, policy_version 15490 (0.0007) [2023-10-12 20:36:09,466][44958] Updated weights for policy 0, policy_version 15410 (0.0008) [2023-10-12 20:36:09,656][44959] Updated weights for policy 1, policy_version 15500 (0.0008) [2023-10-12 20:36:09,846][44958] Updated weights for policy 0, policy_version 15420 (0.0008) [2023-10-12 20:36:10,022][44959] Updated weights for policy 1, policy_version 15510 (0.0010) [2023-10-12 20:36:10,398][44959] Updated weights for policy 1, policy_version 15520 (0.0010) [2023-10-12 20:36:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31686656. Throughput: 0: 1629.9, 1: 1651.0. Samples: 7925676. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 20:36:11,443][43579] Avg episode reward: [(0, '263.560'), (1, '260.480')] [2023-10-12 20:36:13,958][44958] Updated weights for policy 0, policy_version 15430 (0.0007) [2023-10-12 20:36:14,321][44958] Updated weights for policy 0, policy_version 15440 (0.0007) [2023-10-12 20:36:14,507][44959] Updated weights for policy 1, policy_version 15530 (0.0010) [2023-10-12 20:36:14,700][44958] Updated weights for policy 0, policy_version 15450 (0.0008) [2023-10-12 20:36:14,876][44959] Updated weights for policy 1, policy_version 15540 (0.0009) [2023-10-12 20:36:15,241][44959] Updated weights for policy 1, policy_version 15550 (0.0007) [2023-10-12 20:36:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31752192. Throughput: 0: 1633.4, 1: 1662.7. Samples: 7945792. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 20:36:16,443][43579] Avg episode reward: [(0, '259.640'), (1, '253.070')] [2023-10-12 20:36:19,082][44958] Updated weights for policy 0, policy_version 15460 (0.0008) [2023-10-12 20:36:19,362][44959] Updated weights for policy 1, policy_version 15560 (0.0007) [2023-10-12 20:36:19,453][44958] Updated weights for policy 0, policy_version 15470 (0.0007) [2023-10-12 20:36:19,738][44959] Updated weights for policy 1, policy_version 15570 (0.0007) [2023-10-12 20:36:19,818][44958] Updated weights for policy 0, policy_version 15480 (0.0008) [2023-10-12 20:36:20,100][44959] Updated weights for policy 1, policy_version 15580 (0.0010) [2023-10-12 20:36:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31817728. Throughput: 0: 1643.2, 1: 1666.0. Samples: 7956848. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 20:36:21,444][43579] Avg episode reward: [(0, '253.690'), (1, '248.610')] [2023-10-12 20:36:23,918][44958] Updated weights for policy 0, policy_version 15490 (0.0008) [2023-10-12 20:36:24,281][44958] Updated weights for policy 0, policy_version 15500 (0.0010) [2023-10-12 20:36:24,314][44959] Updated weights for policy 1, policy_version 15590 (0.0009) [2023-10-12 20:36:24,658][44958] Updated weights for policy 0, policy_version 15510 (0.0007) [2023-10-12 20:36:24,688][44959] Updated weights for policy 1, policy_version 15600 (0.0009) [2023-10-12 20:36:25,032][44958] Updated weights for policy 0, policy_version 15520 (0.0007) [2023-10-12 20:36:25,065][44959] Updated weights for policy 1, policy_version 15610 (0.0008) [2023-10-12 20:36:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31883264. Throughput: 0: 1635.8, 1: 1644.9. Samples: 7975032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:36:26,443][43579] Avg episode reward: [(0, '256.620'), (1, '253.260')] [2023-10-12 20:36:29,111][44958] Updated weights for policy 0, policy_version 15530 (0.0012) [2023-10-12 20:36:29,276][44959] Updated weights for policy 1, policy_version 15620 (0.0008) [2023-10-12 20:36:29,477][44958] Updated weights for policy 0, policy_version 15540 (0.0008) [2023-10-12 20:36:29,643][44959] Updated weights for policy 1, policy_version 15630 (0.0007) [2023-10-12 20:36:29,848][44958] Updated weights for policy 0, policy_version 15550 (0.0008) [2023-10-12 20:36:30,020][44959] Updated weights for policy 1, policy_version 15640 (0.0008) [2023-10-12 20:36:31,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 31948800. Throughput: 0: 1635.4, 1: 1656.1. Samples: 7995074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:36:31,443][43579] Avg episode reward: [(0, '258.690'), (1, '252.400')] [2023-10-12 20:36:33,974][44959] Updated weights for policy 1, policy_version 15650 (0.0007) [2023-10-12 20:36:34,134][44958] Updated weights for policy 0, policy_version 15560 (0.0009) [2023-10-12 20:36:34,335][44959] Updated weights for policy 1, policy_version 15660 (0.0008) [2023-10-12 20:36:34,510][44958] Updated weights for policy 0, policy_version 15570 (0.0009) [2023-10-12 20:36:34,702][44959] Updated weights for policy 1, policy_version 15670 (0.0008) [2023-10-12 20:36:34,884][44958] Updated weights for policy 0, policy_version 15580 (0.0008) [2023-10-12 20:36:35,069][44959] Updated weights for policy 1, policy_version 15680 (0.0009) [2023-10-12 20:36:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32014336. Throughput: 0: 1640.0, 1: 1654.7. Samples: 8005888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:36:36,443][43579] Avg episode reward: [(0, '258.970'), (1, '249.550')] [2023-10-12 20:36:38,930][44958] Updated weights for policy 0, policy_version 15590 (0.0009) [2023-10-12 20:36:39,151][44959] Updated weights for policy 1, policy_version 15690 (0.0007) [2023-10-12 20:36:39,297][44958] Updated weights for policy 0, policy_version 15600 (0.0010) [2023-10-12 20:36:39,525][44959] Updated weights for policy 1, policy_version 15700 (0.0008) [2023-10-12 20:36:39,669][44958] Updated weights for policy 0, policy_version 15610 (0.0007) [2023-10-12 20:36:39,888][44959] Updated weights for policy 1, policy_version 15710 (0.0007) [2023-10-12 20:36:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32079872. Throughput: 0: 1638.8, 1: 1644.8. Samples: 8024538. Policy #0 lag: (min: 21.0, avg: 22.6, max: 43.0) [2023-10-12 20:36:41,443][43579] Avg episode reward: [(0, '260.410'), (1, '255.920')] [2023-10-12 20:36:43,739][44958] Updated weights for policy 0, policy_version 15620 (0.0009) [2023-10-12 20:36:44,118][44958] Updated weights for policy 0, policy_version 15630 (0.0009) [2023-10-12 20:36:44,170][44959] Updated weights for policy 1, policy_version 15720 (0.0007) [2023-10-12 20:36:44,480][44958] Updated weights for policy 0, policy_version 15640 (0.0010) [2023-10-12 20:36:44,540][44959] Updated weights for policy 1, policy_version 15730 (0.0007) [2023-10-12 20:36:44,907][44959] Updated weights for policy 1, policy_version 15740 (0.0009) [2023-10-12 20:36:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 32145408. Throughput: 0: 1640.1, 1: 1658.5. Samples: 8044776. Policy #0 lag: (min: 21.0, avg: 22.6, max: 43.0) [2023-10-12 20:36:46,444][43579] Avg episode reward: [(0, '267.360'), (1, '252.700')] [2023-10-12 20:36:48,937][44958] Updated weights for policy 0, policy_version 15650 (0.0009) [2023-10-12 20:36:49,233][44959] Updated weights for policy 1, policy_version 15750 (0.0009) [2023-10-12 20:36:49,316][44958] Updated weights for policy 0, policy_version 15660 (0.0007) [2023-10-12 20:36:49,608][44959] Updated weights for policy 1, policy_version 15760 (0.0009) [2023-10-12 20:36:49,687][44958] Updated weights for policy 0, policy_version 15670 (0.0008) [2023-10-12 20:36:49,975][44959] Updated weights for policy 1, policy_version 15770 (0.0009) [2023-10-12 20:36:50,059][44958] Updated weights for policy 0, policy_version 15680 (0.0008) [2023-10-12 20:36:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 32210944. Throughput: 0: 1644.7, 1: 1653.3. Samples: 8055588. Policy #0 lag: (min: 21.0, avg: 22.6, max: 43.0) [2023-10-12 20:36:51,443][43579] Avg episode reward: [(0, '267.020'), (1, '248.240')] [2023-10-12 20:36:53,991][44958] Updated weights for policy 0, policy_version 15690 (0.0010) [2023-10-12 20:36:54,085][44959] Updated weights for policy 1, policy_version 15780 (0.0008) [2023-10-12 20:36:54,350][44958] Updated weights for policy 0, policy_version 15700 (0.0010) [2023-10-12 20:36:54,449][44959] Updated weights for policy 1, policy_version 15790 (0.0007) [2023-10-12 20:36:54,725][44958] Updated weights for policy 0, policy_version 15710 (0.0008) [2023-10-12 20:36:54,822][44959] Updated weights for policy 1, policy_version 15800 (0.0009) [2023-10-12 20:36:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32276480. Throughput: 0: 1652.3, 1: 1647.7. Samples: 8074176. Policy #0 lag: (min: 1.0, avg: 18.3, max: 33.0) [2023-10-12 20:36:56,444][43579] Avg episode reward: [(0, '266.440'), (1, '249.380')] [2023-10-12 20:36:58,805][44958] Updated weights for policy 0, policy_version 15720 (0.0009) [2023-10-12 20:36:59,151][44959] Updated weights for policy 1, policy_version 15810 (0.0009) [2023-10-12 20:36:59,185][44958] Updated weights for policy 0, policy_version 15730 (0.0008) [2023-10-12 20:36:59,527][44959] Updated weights for policy 1, policy_version 15820 (0.0009) [2023-10-12 20:36:59,550][44958] Updated weights for policy 0, policy_version 15740 (0.0008) [2023-10-12 20:36:59,885][44959] Updated weights for policy 1, policy_version 15830 (0.0008) [2023-10-12 20:37:00,250][44959] Updated weights for policy 1, policy_version 15840 (0.0011) [2023-10-12 20:37:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32342016. Throughput: 0: 1650.0, 1: 1642.9. Samples: 8093972. Policy #0 lag: (min: 1.0, avg: 18.3, max: 33.0) [2023-10-12 20:37:01,443][43579] Avg episode reward: [(0, '269.590'), (1, '251.490')] [2023-10-12 20:37:03,926][44958] Updated weights for policy 0, policy_version 15750 (0.0009) [2023-10-12 20:37:04,303][44958] Updated weights for policy 0, policy_version 15760 (0.0009) [2023-10-12 20:37:04,378][44959] Updated weights for policy 1, policy_version 15850 (0.0007) [2023-10-12 20:37:04,668][44958] Updated weights for policy 0, policy_version 15770 (0.0008) [2023-10-12 20:37:04,744][44959] Updated weights for policy 1, policy_version 15860 (0.0009) [2023-10-12 20:37:05,107][44959] Updated weights for policy 1, policy_version 15870 (0.0008) [2023-10-12 20:37:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32407552. Throughput: 0: 1646.3, 1: 1640.6. Samples: 8104758. Policy #0 lag: (min: 1.0, avg: 18.3, max: 33.0) [2023-10-12 20:37:06,444][43579] Avg episode reward: [(0, '271.950'), (1, '247.920')] [2023-10-12 20:37:08,734][44958] Updated weights for policy 0, policy_version 15780 (0.0008) [2023-10-12 20:37:09,102][44958] Updated weights for policy 0, policy_version 15790 (0.0007) [2023-10-12 20:37:09,103][44959] Updated weights for policy 1, policy_version 15880 (0.0009) [2023-10-12 20:37:09,477][44959] Updated weights for policy 1, policy_version 15890 (0.0009) [2023-10-12 20:37:09,480][44958] Updated weights for policy 0, policy_version 15800 (0.0007) [2023-10-12 20:37:09,844][44959] Updated weights for policy 1, policy_version 15900 (0.0008) [2023-10-12 20:37:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32473088. Throughput: 0: 1653.7, 1: 1641.8. Samples: 8123330. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 20:37:11,444][43579] Avg episode reward: [(0, '269.530'), (1, '252.290')] [2023-10-12 20:37:13,655][44958] Updated weights for policy 0, policy_version 15810 (0.0008) [2023-10-12 20:37:14,022][44958] Updated weights for policy 0, policy_version 15820 (0.0008) [2023-10-12 20:37:14,256][44959] Updated weights for policy 1, policy_version 15910 (0.0008) [2023-10-12 20:37:14,392][44958] Updated weights for policy 0, policy_version 15830 (0.0008) [2023-10-12 20:37:14,644][44959] Updated weights for policy 1, policy_version 15920 (0.0007) [2023-10-12 20:37:14,771][44958] Updated weights for policy 0, policy_version 15840 (0.0007) [2023-10-12 20:37:15,011][44959] Updated weights for policy 1, policy_version 15930 (0.0007) [2023-10-12 20:37:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32538624. Throughput: 0: 1643.7, 1: 1646.9. Samples: 8143154. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 20:37:16,443][43579] Avg episode reward: [(0, '274.720'), (1, '248.140')] [2023-10-12 20:37:18,910][44958] Updated weights for policy 0, policy_version 15850 (0.0008) [2023-10-12 20:37:19,107][44959] Updated weights for policy 1, policy_version 15940 (0.0008) [2023-10-12 20:37:19,281][44958] Updated weights for policy 0, policy_version 15860 (0.0010) [2023-10-12 20:37:19,469][44959] Updated weights for policy 1, policy_version 15950 (0.0009) [2023-10-12 20:37:19,648][44958] Updated weights for policy 0, policy_version 15870 (0.0010) [2023-10-12 20:37:19,840][44959] Updated weights for policy 1, policy_version 15960 (0.0010) [2023-10-12 20:37:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32604160. Throughput: 0: 1641.9, 1: 1644.4. Samples: 8153770. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 20:37:21,443][43579] Avg episode reward: [(0, '270.090'), (1, '253.310')] [2023-10-12 20:37:23,779][44958] Updated weights for policy 0, policy_version 15880 (0.0008) [2023-10-12 20:37:23,976][44959] Updated weights for policy 1, policy_version 15970 (0.0009) [2023-10-12 20:37:24,143][44958] Updated weights for policy 0, policy_version 15890 (0.0007) [2023-10-12 20:37:24,347][44959] Updated weights for policy 1, policy_version 15980 (0.0008) [2023-10-12 20:37:24,524][44958] Updated weights for policy 0, policy_version 15900 (0.0009) [2023-10-12 20:37:24,720][44959] Updated weights for policy 1, policy_version 15990 (0.0009) [2023-10-12 20:37:25,091][44959] Updated weights for policy 1, policy_version 16000 (0.0009) [2023-10-12 20:37:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32669696. Throughput: 0: 1646.2, 1: 1642.5. Samples: 8172528. Policy #0 lag: (min: 0.0, avg: 22.5, max: 32.0) [2023-10-12 20:37:26,443][43579] Avg episode reward: [(0, '266.700'), (1, '250.520')] [2023-10-12 20:37:28,639][44958] Updated weights for policy 0, policy_version 15910 (0.0008) [2023-10-12 20:37:29,010][44958] Updated weights for policy 0, policy_version 15920 (0.0009) [2023-10-12 20:37:29,349][44959] Updated weights for policy 1, policy_version 16010 (0.0009) [2023-10-12 20:37:29,384][44958] Updated weights for policy 0, policy_version 15930 (0.0007) [2023-10-12 20:37:29,722][44959] Updated weights for policy 1, policy_version 16020 (0.0009) [2023-10-12 20:37:30,082][44959] Updated weights for policy 1, policy_version 16030 (0.0011) [2023-10-12 20:37:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32735232. Throughput: 0: 1646.2, 1: 1638.4. Samples: 8192580. Policy #0 lag: (min: 0.0, avg: 22.5, max: 32.0) [2023-10-12 20:37:31,443][43579] Avg episode reward: [(0, '265.890'), (1, '255.380')] [2023-10-12 20:37:33,545][44958] Updated weights for policy 0, policy_version 15940 (0.0007) [2023-10-12 20:37:33,908][44958] Updated weights for policy 0, policy_version 15950 (0.0007) [2023-10-12 20:37:34,236][44959] Updated weights for policy 1, policy_version 16040 (0.0008) [2023-10-12 20:37:34,278][44958] Updated weights for policy 0, policy_version 15960 (0.0007) [2023-10-12 20:37:34,603][44959] Updated weights for policy 1, policy_version 16050 (0.0007) [2023-10-12 20:37:34,966][44959] Updated weights for policy 1, policy_version 16060 (0.0008) [2023-10-12 20:37:36,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 32800768. Throughput: 0: 1638.7, 1: 1640.7. Samples: 8203162. Policy #0 lag: (min: 0.0, avg: 22.5, max: 32.0) [2023-10-12 20:37:36,444][43579] Avg episode reward: [(0, '264.440'), (1, '266.030')] [2023-10-12 20:37:38,465][44958] Updated weights for policy 0, policy_version 15970 (0.0007) [2023-10-12 20:37:38,831][44958] Updated weights for policy 0, policy_version 15980 (0.0007) [2023-10-12 20:37:39,182][44959] Updated weights for policy 1, policy_version 16070 (0.0010) [2023-10-12 20:37:39,207][44958] Updated weights for policy 0, policy_version 15990 (0.0007) [2023-10-12 20:37:39,548][44959] Updated weights for policy 1, policy_version 16080 (0.0008) [2023-10-12 20:37:39,584][44958] Updated weights for policy 0, policy_version 16000 (0.0008) [2023-10-12 20:37:39,913][44959] Updated weights for policy 1, policy_version 16090 (0.0009) [2023-10-12 20:37:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32866304. Throughput: 0: 1647.8, 1: 1634.7. Samples: 8221888. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-12 20:37:41,444][43579] Avg episode reward: [(0, '257.260'), (1, '267.950')] [2023-10-12 20:37:41,445][44583] Saving new best policy, reward=267.950! [2023-10-12 20:37:43,720][44958] Updated weights for policy 0, policy_version 16010 (0.0008) [2023-10-12 20:37:44,100][44958] Updated weights for policy 0, policy_version 16020 (0.0007) [2023-10-12 20:37:44,119][44959] Updated weights for policy 1, policy_version 16100 (0.0008) [2023-10-12 20:37:44,475][44958] Updated weights for policy 0, policy_version 16030 (0.0008) [2023-10-12 20:37:44,482][44959] Updated weights for policy 1, policy_version 16110 (0.0009) [2023-10-12 20:37:44,856][44959] Updated weights for policy 1, policy_version 16120 (0.0009) [2023-10-12 20:37:46,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32931840. Throughput: 0: 1647.0, 1: 1642.3. Samples: 8241992. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-12 20:37:46,444][43579] Avg episode reward: [(0, '261.260'), (1, '268.840')] [2023-10-12 20:37:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000016032_16416768.pth... [2023-10-12 20:37:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000016128_16515072.pth... [2023-10-12 20:37:46,484][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000014496_14843904.pth [2023-10-12 20:37:46,491][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000014592_14942208.pth [2023-10-12 20:37:46,495][44583] Saving new best policy, reward=268.840! [2023-10-12 20:37:48,693][44958] Updated weights for policy 0, policy_version 16040 (0.0009) [2023-10-12 20:37:49,061][44958] Updated weights for policy 0, policy_version 16050 (0.0008) [2023-10-12 20:37:49,070][44959] Updated weights for policy 1, policy_version 16130 (0.0008) [2023-10-12 20:37:49,432][44958] Updated weights for policy 0, policy_version 16060 (0.0007) [2023-10-12 20:37:49,443][44959] Updated weights for policy 1, policy_version 16140 (0.0009) [2023-10-12 20:37:49,813][44959] Updated weights for policy 1, policy_version 16150 (0.0010) [2023-10-12 20:37:50,187][44959] Updated weights for policy 1, policy_version 16160 (0.0010) [2023-10-12 20:37:51,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32997376. Throughput: 0: 1640.5, 1: 1640.7. Samples: 8252412. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-12 20:37:51,443][43579] Avg episode reward: [(0, '259.800'), (1, '257.170')] [2023-10-12 20:37:53,651][44958] Updated weights for policy 0, policy_version 16070 (0.0009) [2023-10-12 20:37:54,022][44958] Updated weights for policy 0, policy_version 16080 (0.0010) [2023-10-12 20:37:54,395][44958] Updated weights for policy 0, policy_version 16090 (0.0009) [2023-10-12 20:37:54,395][44959] Updated weights for policy 1, policy_version 16170 (0.0008) [2023-10-12 20:37:54,766][44959] Updated weights for policy 1, policy_version 16180 (0.0007) [2023-10-12 20:37:55,139][44959] Updated weights for policy 1, policy_version 16190 (0.0008) [2023-10-12 20:37:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33062912. Throughput: 0: 1648.5, 1: 1638.2. Samples: 8271228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:37:56,443][43579] Avg episode reward: [(0, '263.150'), (1, '251.390')] [2023-10-12 20:37:58,510][44958] Updated weights for policy 0, policy_version 16100 (0.0009) [2023-10-12 20:37:58,880][44958] Updated weights for policy 0, policy_version 16110 (0.0007) [2023-10-12 20:37:59,253][44958] Updated weights for policy 0, policy_version 16120 (0.0008) [2023-10-12 20:37:59,455][44959] Updated weights for policy 1, policy_version 16200 (0.0008) [2023-10-12 20:37:59,816][44959] Updated weights for policy 1, policy_version 16210 (0.0011) [2023-10-12 20:38:00,195][44959] Updated weights for policy 1, policy_version 16220 (0.0009) [2023-10-12 20:38:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 33128448. Throughput: 0: 1653.8, 1: 1636.9. Samples: 8291234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:38:01,444][43579] Avg episode reward: [(0, '259.780'), (1, '247.460')] [2023-10-12 20:38:03,380][44958] Updated weights for policy 0, policy_version 16130 (0.0007) [2023-10-12 20:38:03,770][44958] Updated weights for policy 0, policy_version 16140 (0.0009) [2023-10-12 20:38:04,133][44958] Updated weights for policy 0, policy_version 16150 (0.0009) [2023-10-12 20:38:04,250][44959] Updated weights for policy 1, policy_version 16230 (0.0008) [2023-10-12 20:38:04,512][44958] Updated weights for policy 0, policy_version 16160 (0.0007) [2023-10-12 20:38:04,617][44959] Updated weights for policy 1, policy_version 16240 (0.0009) [2023-10-12 20:38:04,992][44959] Updated weights for policy 1, policy_version 16250 (0.0007) [2023-10-12 20:38:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33193984. Throughput: 0: 1651.1, 1: 1638.4. Samples: 8301800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:38:06,443][43579] Avg episode reward: [(0, '256.200'), (1, '238.510')] [2023-10-12 20:38:08,542][44958] Updated weights for policy 0, policy_version 16170 (0.0011) [2023-10-12 20:38:08,918][44958] Updated weights for policy 0, policy_version 16180 (0.0010) [2023-10-12 20:38:09,252][44959] Updated weights for policy 1, policy_version 16260 (0.0009) [2023-10-12 20:38:09,285][44958] Updated weights for policy 0, policy_version 16190 (0.0008) [2023-10-12 20:38:09,624][44959] Updated weights for policy 1, policy_version 16270 (0.0007) [2023-10-12 20:38:09,989][44959] Updated weights for policy 1, policy_version 16280 (0.0008) [2023-10-12 20:38:11,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33259520. Throughput: 0: 1652.7, 1: 1636.8. Samples: 8320556. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) [2023-10-12 20:38:11,443][43579] Avg episode reward: [(0, '258.010'), (1, '241.810')] [2023-10-12 20:38:13,248][44958] Updated weights for policy 0, policy_version 16200 (0.0009) [2023-10-12 20:38:13,620][44958] Updated weights for policy 0, policy_version 16210 (0.0008) [2023-10-12 20:38:13,988][44958] Updated weights for policy 0, policy_version 16220 (0.0008) [2023-10-12 20:38:14,181][44959] Updated weights for policy 1, policy_version 16290 (0.0010) [2023-10-12 20:38:14,540][44959] Updated weights for policy 1, policy_version 16300 (0.0007) [2023-10-12 20:38:14,911][44959] Updated weights for policy 1, policy_version 16310 (0.0008) [2023-10-12 20:38:15,276][44959] Updated weights for policy 1, policy_version 16320 (0.0008) [2023-10-12 20:38:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33325056. Throughput: 0: 1659.8, 1: 1635.5. Samples: 8340866. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) [2023-10-12 20:38:16,444][43579] Avg episode reward: [(0, '259.780'), (1, '236.620')] [2023-10-12 20:38:18,130][44958] Updated weights for policy 0, policy_version 16230 (0.0010) [2023-10-12 20:38:18,501][44958] Updated weights for policy 0, policy_version 16240 (0.0011) [2023-10-12 20:38:18,872][44958] Updated weights for policy 0, policy_version 16250 (0.0009) [2023-10-12 20:38:19,395][44959] Updated weights for policy 1, policy_version 16330 (0.0008) [2023-10-12 20:38:19,751][44959] Updated weights for policy 1, policy_version 16340 (0.0009) [2023-10-12 20:38:20,122][44959] Updated weights for policy 1, policy_version 16350 (0.0009) [2023-10-12 20:38:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33390592. Throughput: 0: 1646.9, 1: 1640.3. Samples: 8351086. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) [2023-10-12 20:38:21,443][43579] Avg episode reward: [(0, '259.080'), (1, '240.150')] [2023-10-12 20:38:23,022][44958] Updated weights for policy 0, policy_version 16260 (0.0008) [2023-10-12 20:38:23,393][44958] Updated weights for policy 0, policy_version 16270 (0.0008) [2023-10-12 20:38:23,759][44958] Updated weights for policy 0, policy_version 16280 (0.0007) [2023-10-12 20:38:24,208][44959] Updated weights for policy 1, policy_version 16360 (0.0008) [2023-10-12 20:38:24,584][44959] Updated weights for policy 1, policy_version 16370 (0.0007) [2023-10-12 20:38:24,953][44959] Updated weights for policy 1, policy_version 16380 (0.0009) [2023-10-12 20:38:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 33456128. Throughput: 0: 1648.2, 1: 1644.6. Samples: 8370062. Policy #0 lag: (min: 27.0, avg: 33.6, max: 59.0) [2023-10-12 20:38:26,444][43579] Avg episode reward: [(0, '249.670'), (1, '236.850')] [2023-10-12 20:38:28,065][44958] Updated weights for policy 0, policy_version 16290 (0.0009) [2023-10-12 20:38:28,434][44958] Updated weights for policy 0, policy_version 16300 (0.0010) [2023-10-12 20:38:28,823][44958] Updated weights for policy 0, policy_version 16310 (0.0012) [2023-10-12 20:38:29,129][44959] Updated weights for policy 1, policy_version 16390 (0.0010) [2023-10-12 20:38:29,182][44958] Updated weights for policy 0, policy_version 16320 (0.0008) [2023-10-12 20:38:29,499][44959] Updated weights for policy 1, policy_version 16400 (0.0009) [2023-10-12 20:38:29,858][44959] Updated weights for policy 1, policy_version 16410 (0.0008) [2023-10-12 20:38:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33521664. Throughput: 0: 1647.5, 1: 1643.0. Samples: 8390064. Policy #0 lag: (min: 27.0, avg: 33.6, max: 59.0) [2023-10-12 20:38:31,443][43579] Avg episode reward: [(0, '253.520'), (1, '245.910')] [2023-10-12 20:38:33,384][44958] Updated weights for policy 0, policy_version 16330 (0.0008) [2023-10-12 20:38:33,761][44958] Updated weights for policy 0, policy_version 16340 (0.0010) [2023-10-12 20:38:33,835][44959] Updated weights for policy 1, policy_version 16420 (0.0007) [2023-10-12 20:38:34,135][44958] Updated weights for policy 0, policy_version 16350 (0.0008) [2023-10-12 20:38:34,207][44959] Updated weights for policy 1, policy_version 16430 (0.0008) [2023-10-12 20:38:34,576][44959] Updated weights for policy 1, policy_version 16440 (0.0010) [2023-10-12 20:38:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33587200. Throughput: 0: 1640.1, 1: 1645.0. Samples: 8400242. Policy #0 lag: (min: 27.0, avg: 33.6, max: 59.0) [2023-10-12 20:38:36,444][43579] Avg episode reward: [(0, '253.140'), (1, '243.440')] [2023-10-12 20:38:38,142][44958] Updated weights for policy 0, policy_version 16360 (0.0008) [2023-10-12 20:38:38,522][44958] Updated weights for policy 0, policy_version 16370 (0.0007) [2023-10-12 20:38:38,805][44959] Updated weights for policy 1, policy_version 16450 (0.0008) [2023-10-12 20:38:38,900][44958] Updated weights for policy 0, policy_version 16380 (0.0007) [2023-10-12 20:38:39,179][44959] Updated weights for policy 1, policy_version 16460 (0.0007) [2023-10-12 20:38:39,545][44959] Updated weights for policy 1, policy_version 16470 (0.0008) [2023-10-12 20:38:39,912][44959] Updated weights for policy 1, policy_version 16480 (0.0009) [2023-10-12 20:38:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33652736. Throughput: 0: 1649.6, 1: 1647.2. Samples: 8419586. Policy #0 lag: (min: 8.0, avg: 35.7, max: 40.0) [2023-10-12 20:38:41,444][43579] Avg episode reward: [(0, '251.900'), (1, '256.860')] [2023-10-12 20:38:43,104][44958] Updated weights for policy 0, policy_version 16390 (0.0008) [2023-10-12 20:38:43,470][44958] Updated weights for policy 0, policy_version 16400 (0.0007) [2023-10-12 20:38:43,846][44958] Updated weights for policy 0, policy_version 16410 (0.0009) [2023-10-12 20:38:44,198][44959] Updated weights for policy 1, policy_version 16490 (0.0007) [2023-10-12 20:38:44,568][44959] Updated weights for policy 1, policy_version 16500 (0.0007) [2023-10-12 20:38:44,932][44959] Updated weights for policy 1, policy_version 16510 (0.0007) [2023-10-12 20:38:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33718272. Throughput: 0: 1647.4, 1: 1652.9. Samples: 8439744. Policy #0 lag: (min: 8.0, avg: 35.7, max: 40.0) [2023-10-12 20:38:46,443][43579] Avg episode reward: [(0, '249.220'), (1, '256.740')] [2023-10-12 20:38:47,956][44958] Updated weights for policy 0, policy_version 16420 (0.0010) [2023-10-12 20:38:48,344][44958] Updated weights for policy 0, policy_version 16430 (0.0009) [2023-10-12 20:38:48,705][44958] Updated weights for policy 0, policy_version 16440 (0.0009) [2023-10-12 20:38:48,883][44959] Updated weights for policy 1, policy_version 16520 (0.0009) [2023-10-12 20:38:49,260][44959] Updated weights for policy 1, policy_version 16530 (0.0010) [2023-10-12 20:38:49,629][44959] Updated weights for policy 1, policy_version 16540 (0.0007) [2023-10-12 20:38:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 33783808. Throughput: 0: 1635.1, 1: 1643.0. Samples: 8449314. Policy #0 lag: (min: 8.0, avg: 35.7, max: 40.0) [2023-10-12 20:38:51,443][43579] Avg episode reward: [(0, '249.710'), (1, '255.200')] [2023-10-12 20:38:52,945][44958] Updated weights for policy 0, policy_version 16450 (0.0008) [2023-10-12 20:38:53,314][44958] Updated weights for policy 0, policy_version 16460 (0.0010) [2023-10-12 20:38:53,691][44958] Updated weights for policy 0, policy_version 16470 (0.0009) [2023-10-12 20:38:53,755][44959] Updated weights for policy 1, policy_version 16550 (0.0008) [2023-10-12 20:38:54,059][44958] Updated weights for policy 0, policy_version 16480 (0.0008) [2023-10-12 20:38:54,118][44959] Updated weights for policy 1, policy_version 16560 (0.0009) [2023-10-12 20:38:54,484][44959] Updated weights for policy 1, policy_version 16570 (0.0009) [2023-10-12 20:38:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 33849344. Throughput: 0: 1644.0, 1: 1649.1. Samples: 8468744. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 20:38:56,443][43579] Avg episode reward: [(0, '259.090'), (1, '258.610')] [2023-10-12 20:38:58,370][44958] Updated weights for policy 0, policy_version 16490 (0.0011) [2023-10-12 20:38:58,703][44959] Updated weights for policy 1, policy_version 16580 (0.0009) [2023-10-12 20:38:58,748][44958] Updated weights for policy 0, policy_version 16500 (0.0008) [2023-10-12 20:38:59,067][44959] Updated weights for policy 1, policy_version 16590 (0.0007) [2023-10-12 20:38:59,111][44958] Updated weights for policy 0, policy_version 16510 (0.0009) [2023-10-12 20:38:59,426][44959] Updated weights for policy 1, policy_version 16600 (0.0009) [2023-10-12 20:39:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 33914880. Throughput: 0: 1636.0, 1: 1658.4. Samples: 8489116. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 20:39:01,444][43579] Avg episode reward: [(0, '257.690'), (1, '259.100')] [2023-10-12 20:39:03,265][44958] Updated weights for policy 0, policy_version 16520 (0.0008) [2023-10-12 20:39:03,509][44959] Updated weights for policy 1, policy_version 16610 (0.0009) [2023-10-12 20:39:03,643][44958] Updated weights for policy 0, policy_version 16530 (0.0007) [2023-10-12 20:39:03,877][44959] Updated weights for policy 1, policy_version 16620 (0.0009) [2023-10-12 20:39:04,005][44958] Updated weights for policy 0, policy_version 16540 (0.0008) [2023-10-12 20:39:04,249][44959] Updated weights for policy 1, policy_version 16630 (0.0008) [2023-10-12 20:39:04,620][44959] Updated weights for policy 1, policy_version 16640 (0.0009) [2023-10-12 20:39:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 33980416. Throughput: 0: 1638.2, 1: 1641.6. Samples: 8498680. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 20:39:06,444][43579] Avg episode reward: [(0, '258.730'), (1, '258.490')] [2023-10-12 20:39:08,229][44958] Updated weights for policy 0, policy_version 16550 (0.0008) [2023-10-12 20:39:08,605][44958] Updated weights for policy 0, policy_version 16560 (0.0007) [2023-10-12 20:39:08,855][44959] Updated weights for policy 1, policy_version 16650 (0.0007) [2023-10-12 20:39:08,981][44958] Updated weights for policy 0, policy_version 16570 (0.0008) [2023-10-12 20:39:09,227][44959] Updated weights for policy 1, policy_version 16660 (0.0011) [2023-10-12 20:39:09,606][44959] Updated weights for policy 1, policy_version 16670 (0.0009) [2023-10-12 20:39:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34045952. Throughput: 0: 1644.0, 1: 1645.1. Samples: 8518072. Policy #0 lag: (min: 9.0, avg: 18.1, max: 41.0) [2023-10-12 20:39:11,444][43579] Avg episode reward: [(0, '258.690'), (1, '256.360')] [2023-10-12 20:39:12,796][44958] Updated weights for policy 0, policy_version 16580 (0.0007) [2023-10-12 20:39:13,176][44958] Updated weights for policy 0, policy_version 16590 (0.0008) [2023-10-12 20:39:13,555][44958] Updated weights for policy 0, policy_version 16600 (0.0008) [2023-10-12 20:39:13,803][44959] Updated weights for policy 1, policy_version 16680 (0.0008) [2023-10-12 20:39:14,176][44959] Updated weights for policy 1, policy_version 16690 (0.0008) [2023-10-12 20:39:14,542][44959] Updated weights for policy 1, policy_version 16700 (0.0007) [2023-10-12 20:39:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34111488. Throughput: 0: 1645.0, 1: 1652.0. Samples: 8538428. Policy #0 lag: (min: 9.0, avg: 18.1, max: 41.0) [2023-10-12 20:39:16,444][43579] Avg episode reward: [(0, '262.350'), (1, '257.350')] [2023-10-12 20:39:17,838][44958] Updated weights for policy 0, policy_version 16610 (0.0008) [2023-10-12 20:39:18,208][44958] Updated weights for policy 0, policy_version 16620 (0.0010) [2023-10-12 20:39:18,581][44958] Updated weights for policy 0, policy_version 16630 (0.0008) [2023-10-12 20:39:18,731][44959] Updated weights for policy 1, policy_version 16710 (0.0008) [2023-10-12 20:39:18,950][44958] Updated weights for policy 0, policy_version 16640 (0.0010) [2023-10-12 20:39:19,106][44959] Updated weights for policy 1, policy_version 16720 (0.0009) [2023-10-12 20:39:19,474][44959] Updated weights for policy 1, policy_version 16730 (0.0010) [2023-10-12 20:39:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34177024. Throughput: 0: 1639.9, 1: 1638.7. Samples: 8547778. Policy #0 lag: (min: 9.0, avg: 18.1, max: 41.0) [2023-10-12 20:39:21,444][43579] Avg episode reward: [(0, '262.410'), (1, '257.080')] [2023-10-12 20:39:23,123][44958] Updated weights for policy 0, policy_version 16650 (0.0007) [2023-10-12 20:39:23,499][44958] Updated weights for policy 0, policy_version 16660 (0.0007) [2023-10-12 20:39:23,652][44959] Updated weights for policy 1, policy_version 16740 (0.0010) [2023-10-12 20:39:23,877][44958] Updated weights for policy 0, policy_version 16670 (0.0008) [2023-10-12 20:39:24,016][44959] Updated weights for policy 1, policy_version 16750 (0.0007) [2023-10-12 20:39:24,376][44959] Updated weights for policy 1, policy_version 16760 (0.0008) [2023-10-12 20:39:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34242560. Throughput: 0: 1644.8, 1: 1645.2. Samples: 8567638. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 20:39:26,443][43579] Avg episode reward: [(0, '263.310'), (1, '260.970')] [2023-10-12 20:39:28,026][44958] Updated weights for policy 0, policy_version 16680 (0.0009) [2023-10-12 20:39:28,392][44958] Updated weights for policy 0, policy_version 16690 (0.0008) [2023-10-12 20:39:28,723][44959] Updated weights for policy 1, policy_version 16770 (0.0009) [2023-10-12 20:39:28,769][44958] Updated weights for policy 0, policy_version 16700 (0.0008) [2023-10-12 20:39:29,137][44959] Updated weights for policy 1, policy_version 16780 (0.0009) [2023-10-12 20:39:29,501][44959] Updated weights for policy 1, policy_version 16790 (0.0008) [2023-10-12 20:39:29,867][44959] Updated weights for policy 1, policy_version 16800 (0.0007) [2023-10-12 20:39:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34308096. Throughput: 0: 1646.1, 1: 1640.1. Samples: 8587624. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 20:39:31,444][43579] Avg episode reward: [(0, '260.090'), (1, '259.840')] [2023-10-12 20:39:32,995][44958] Updated weights for policy 0, policy_version 16710 (0.0009) [2023-10-12 20:39:33,367][44958] Updated weights for policy 0, policy_version 16720 (0.0011) [2023-10-12 20:39:33,737][44958] Updated weights for policy 0, policy_version 16730 (0.0009) [2023-10-12 20:39:34,033][44959] Updated weights for policy 1, policy_version 16810 (0.0007) [2023-10-12 20:39:34,410][44959] Updated weights for policy 1, policy_version 16820 (0.0007) [2023-10-12 20:39:34,783][44959] Updated weights for policy 1, policy_version 16830 (0.0007) [2023-10-12 20:39:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34373632. Throughput: 0: 1646.5, 1: 1645.2. Samples: 8597444. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 20:39:36,444][43579] Avg episode reward: [(0, '252.660'), (1, '259.690')] [2023-10-12 20:39:37,958][44958] Updated weights for policy 0, policy_version 16740 (0.0008) [2023-10-12 20:39:38,350][44958] Updated weights for policy 0, policy_version 16750 (0.0009) [2023-10-12 20:39:38,725][44958] Updated weights for policy 0, policy_version 16760 (0.0010) [2023-10-12 20:39:38,954][44959] Updated weights for policy 1, policy_version 16840 (0.0009) [2023-10-12 20:39:39,322][44959] Updated weights for policy 1, policy_version 16850 (0.0009) [2023-10-12 20:39:39,699][44959] Updated weights for policy 1, policy_version 16860 (0.0007) [2023-10-12 20:39:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34439168. Throughput: 0: 1645.3, 1: 1644.0. Samples: 8616762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:39:41,444][43579] Avg episode reward: [(0, '246.360'), (1, '261.870')] [2023-10-12 20:39:42,762][44958] Updated weights for policy 0, policy_version 16770 (0.0009) [2023-10-12 20:39:43,142][44958] Updated weights for policy 0, policy_version 16780 (0.0008) [2023-10-12 20:39:43,519][44958] Updated weights for policy 0, policy_version 16790 (0.0008) [2023-10-12 20:39:43,749][44959] Updated weights for policy 1, policy_version 16870 (0.0007) [2023-10-12 20:39:43,876][44958] Updated weights for policy 0, policy_version 16800 (0.0009) [2023-10-12 20:39:44,123][44959] Updated weights for policy 1, policy_version 16880 (0.0008) [2023-10-12 20:39:44,497][44959] Updated weights for policy 1, policy_version 16890 (0.0008) [2023-10-12 20:39:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34504704. Throughput: 0: 1644.9, 1: 1640.8. Samples: 8636970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:39:46,443][43579] Avg episode reward: [(0, '245.700'), (1, '262.620')] [2023-10-12 20:39:46,453][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000016800_17203200.pth... [2023-10-12 20:39:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000016896_17301504.pth... [2023-10-12 20:39:46,489][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000015360_15728640.pth [2023-10-12 20:39:46,490][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000015264_15630336.pth [2023-10-12 20:39:48,003][44958] Updated weights for policy 0, policy_version 16810 (0.0007) [2023-10-12 20:39:48,367][44958] Updated weights for policy 0, policy_version 16820 (0.0008) [2023-10-12 20:39:48,698][44959] Updated weights for policy 1, policy_version 16900 (0.0009) [2023-10-12 20:39:48,749][44958] Updated weights for policy 0, policy_version 16830 (0.0007) [2023-10-12 20:39:49,059][44959] Updated weights for policy 1, policy_version 16910 (0.0007) [2023-10-12 20:39:49,421][44959] Updated weights for policy 1, policy_version 16920 (0.0009) [2023-10-12 20:39:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34570240. Throughput: 0: 1641.5, 1: 1641.8. Samples: 8646428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:39:51,443][43579] Avg episode reward: [(0, '248.370'), (1, '265.730')] [2023-10-12 20:39:53,238][44958] Updated weights for policy 0, policy_version 16840 (0.0010) [2023-10-12 20:39:53,482][44959] Updated weights for policy 1, policy_version 16930 (0.0007) [2023-10-12 20:39:53,611][44958] Updated weights for policy 0, policy_version 16850 (0.0007) [2023-10-12 20:39:53,845][44959] Updated weights for policy 1, policy_version 16940 (0.0008) [2023-10-12 20:39:53,974][44958] Updated weights for policy 0, policy_version 16860 (0.0009) [2023-10-12 20:39:54,221][44959] Updated weights for policy 1, policy_version 16950 (0.0008) [2023-10-12 20:39:54,585][44959] Updated weights for policy 1, policy_version 16960 (0.0011) [2023-10-12 20:39:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34635776. Throughput: 0: 1638.7, 1: 1646.6. Samples: 8665910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:39:56,443][43579] Avg episode reward: [(0, '251.370'), (1, '265.570')] [2023-10-12 20:39:57,878][44958] Updated weights for policy 0, policy_version 16870 (0.0010) [2023-10-12 20:39:58,253][44958] Updated weights for policy 0, policy_version 16880 (0.0010) [2023-10-12 20:39:58,625][44958] Updated weights for policy 0, policy_version 16890 (0.0010) [2023-10-12 20:39:58,794][44959] Updated weights for policy 1, policy_version 16970 (0.0008) [2023-10-12 20:39:59,161][44959] Updated weights for policy 1, policy_version 16980 (0.0010) [2023-10-12 20:39:59,534][44959] Updated weights for policy 1, policy_version 16990 (0.0008) [2023-10-12 20:40:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34701312. Throughput: 0: 1641.5, 1: 1644.9. Samples: 8686316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:40:01,443][43579] Avg episode reward: [(0, '245.240'), (1, '263.880')] [2023-10-12 20:40:02,805][44958] Updated weights for policy 0, policy_version 16900 (0.0009) [2023-10-12 20:40:03,183][44958] Updated weights for policy 0, policy_version 16910 (0.0007) [2023-10-12 20:40:03,551][44958] Updated weights for policy 0, policy_version 16920 (0.0009) [2023-10-12 20:40:03,870][44959] Updated weights for policy 1, policy_version 17000 (0.0009) [2023-10-12 20:40:04,248][44959] Updated weights for policy 1, policy_version 17010 (0.0007) [2023-10-12 20:40:04,617][44959] Updated weights for policy 1, policy_version 17020 (0.0007) [2023-10-12 20:40:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34766848. Throughput: 0: 1642.3, 1: 1644.4. Samples: 8695676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:40:06,444][43579] Avg episode reward: [(0, '257.980'), (1, '268.880')] [2023-10-12 20:40:06,445][44583] Saving new best policy, reward=268.880! [2023-10-12 20:40:07,799][44958] Updated weights for policy 0, policy_version 16930 (0.0009) [2023-10-12 20:40:08,173][44958] Updated weights for policy 0, policy_version 16940 (0.0007) [2023-10-12 20:40:08,533][44959] Updated weights for policy 1, policy_version 17030 (0.0007) [2023-10-12 20:40:08,546][44958] Updated weights for policy 0, policy_version 16950 (0.0007) [2023-10-12 20:40:08,901][44959] Updated weights for policy 1, policy_version 17040 (0.0008) [2023-10-12 20:40:08,921][44958] Updated weights for policy 0, policy_version 16960 (0.0007) [2023-10-12 20:40:09,268][44959] Updated weights for policy 1, policy_version 17050 (0.0008) [2023-10-12 20:40:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34832384. Throughput: 0: 1638.4, 1: 1646.6. Samples: 8715460. Policy #0 lag: (min: 12.0, avg: 17.2, max: 44.0) [2023-10-12 20:40:11,444][43579] Avg episode reward: [(0, '259.600'), (1, '263.560')] [2023-10-12 20:40:12,949][44958] Updated weights for policy 0, policy_version 16970 (0.0009) [2023-10-12 20:40:13,321][44958] Updated weights for policy 0, policy_version 16980 (0.0007) [2023-10-12 20:40:13,329][44959] Updated weights for policy 1, policy_version 17060 (0.0008) [2023-10-12 20:40:13,682][44958] Updated weights for policy 0, policy_version 16990 (0.0009) [2023-10-12 20:40:13,700][44959] Updated weights for policy 1, policy_version 17070 (0.0008) [2023-10-12 20:40:14,060][44959] Updated weights for policy 1, policy_version 17080 (0.0008) [2023-10-12 20:40:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34897920. Throughput: 0: 1639.3, 1: 1653.1. Samples: 8735784. Policy #0 lag: (min: 12.0, avg: 17.2, max: 44.0) [2023-10-12 20:40:16,443][43579] Avg episode reward: [(0, '263.370'), (1, '262.320')] [2023-10-12 20:40:17,956][44958] Updated weights for policy 0, policy_version 17000 (0.0008) [2023-10-12 20:40:18,323][44958] Updated weights for policy 0, policy_version 17010 (0.0010) [2023-10-12 20:40:18,345][44959] Updated weights for policy 1, policy_version 17090 (0.0007) [2023-10-12 20:40:18,704][44958] Updated weights for policy 0, policy_version 17020 (0.0010) [2023-10-12 20:40:18,717][44959] Updated weights for policy 1, policy_version 17100 (0.0008) [2023-10-12 20:40:19,083][44959] Updated weights for policy 1, policy_version 17110 (0.0009) [2023-10-12 20:40:19,452][44959] Updated weights for policy 1, policy_version 17120 (0.0011) [2023-10-12 20:40:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 34963456. Throughput: 0: 1642.1, 1: 1639.4. Samples: 8745112. Policy #0 lag: (min: 12.0, avg: 17.2, max: 44.0) [2023-10-12 20:40:21,444][43579] Avg episode reward: [(0, '264.250'), (1, '255.280')] [2023-10-12 20:40:22,695][44958] Updated weights for policy 0, policy_version 17030 (0.0008) [2023-10-12 20:40:23,060][44958] Updated weights for policy 0, policy_version 17040 (0.0009) [2023-10-12 20:40:23,437][44958] Updated weights for policy 0, policy_version 17050 (0.0007) [2023-10-12 20:40:23,492][44959] Updated weights for policy 1, policy_version 17130 (0.0008) [2023-10-12 20:40:23,861][44959] Updated weights for policy 1, policy_version 17140 (0.0011) [2023-10-12 20:40:24,236][44959] Updated weights for policy 1, policy_version 17150 (0.0008) [2023-10-12 20:40:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35028992. Throughput: 0: 1650.1, 1: 1645.3. Samples: 8765056. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 20:40:26,443][43579] Avg episode reward: [(0, '268.780'), (1, '255.380')] [2023-10-12 20:40:27,849][44958] Updated weights for policy 0, policy_version 17060 (0.0007) [2023-10-12 20:40:28,233][44958] Updated weights for policy 0, policy_version 17070 (0.0007) [2023-10-12 20:40:28,494][44959] Updated weights for policy 1, policy_version 17160 (0.0008) [2023-10-12 20:40:28,613][44958] Updated weights for policy 0, policy_version 17080 (0.0008) [2023-10-12 20:40:28,852][44959] Updated weights for policy 1, policy_version 17170 (0.0008) [2023-10-12 20:40:29,224][44959] Updated weights for policy 1, policy_version 17180 (0.0010) [2023-10-12 20:40:31,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35094528. Throughput: 0: 1642.1, 1: 1649.0. Samples: 8785070. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 20:40:31,444][43579] Avg episode reward: [(0, '280.380'), (1, '251.740')] [2023-10-12 20:40:31,455][44518] Saving new best policy, reward=280.380! [2023-10-12 20:40:32,795][44958] Updated weights for policy 0, policy_version 17090 (0.0009) [2023-10-12 20:40:33,168][44958] Updated weights for policy 0, policy_version 17100 (0.0009) [2023-10-12 20:40:33,326][44959] Updated weights for policy 1, policy_version 17190 (0.0010) [2023-10-12 20:40:33,543][44958] Updated weights for policy 0, policy_version 17110 (0.0009) [2023-10-12 20:40:33,688][44959] Updated weights for policy 1, policy_version 17200 (0.0008) [2023-10-12 20:40:33,911][44958] Updated weights for policy 0, policy_version 17120 (0.0008) [2023-10-12 20:40:34,067][44959] Updated weights for policy 1, policy_version 17210 (0.0008) [2023-10-12 20:40:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35160064. Throughput: 0: 1642.4, 1: 1642.2. Samples: 8794236. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 20:40:36,444][43579] Avg episode reward: [(0, '271.810'), (1, '248.920')] [2023-10-12 20:40:38,072][44958] Updated weights for policy 0, policy_version 17130 (0.0007) [2023-10-12 20:40:38,397][44959] Updated weights for policy 1, policy_version 17220 (0.0008) [2023-10-12 20:40:38,434][44958] Updated weights for policy 0, policy_version 17140 (0.0009) [2023-10-12 20:40:38,768][44959] Updated weights for policy 1, policy_version 17230 (0.0008) [2023-10-12 20:40:38,813][44958] Updated weights for policy 0, policy_version 17150 (0.0007) [2023-10-12 20:40:39,131][44959] Updated weights for policy 1, policy_version 17240 (0.0008) [2023-10-12 20:40:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35225600. Throughput: 0: 1646.4, 1: 1645.9. Samples: 8814064. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:40:41,444][43579] Avg episode reward: [(0, '268.140'), (1, '254.860')] [2023-10-12 20:40:43,128][44958] Updated weights for policy 0, policy_version 17160 (0.0007) [2023-10-12 20:40:43,289][44959] Updated weights for policy 1, policy_version 17250 (0.0009) [2023-10-12 20:40:43,500][44958] Updated weights for policy 0, policy_version 17170 (0.0010) [2023-10-12 20:40:43,654][44959] Updated weights for policy 1, policy_version 17260 (0.0007) [2023-10-12 20:40:43,876][44958] Updated weights for policy 0, policy_version 17180 (0.0009) [2023-10-12 20:40:44,021][44959] Updated weights for policy 1, policy_version 17270 (0.0009) [2023-10-12 20:40:44,394][44959] Updated weights for policy 1, policy_version 17280 (0.0009) [2023-10-12 20:40:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35291136. Throughput: 0: 1639.2, 1: 1649.2. Samples: 8834294. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:40:46,443][43579] Avg episode reward: [(0, '264.590'), (1, '260.370')] [2023-10-12 20:40:48,069][44958] Updated weights for policy 0, policy_version 17190 (0.0009) [2023-10-12 20:40:48,433][44958] Updated weights for policy 0, policy_version 17200 (0.0009) [2023-10-12 20:40:48,532][44959] Updated weights for policy 1, policy_version 17290 (0.0008) [2023-10-12 20:40:48,808][44958] Updated weights for policy 0, policy_version 17210 (0.0008) [2023-10-12 20:40:48,902][44959] Updated weights for policy 1, policy_version 17300 (0.0009) [2023-10-12 20:40:49,267][44959] Updated weights for policy 1, policy_version 17310 (0.0009) [2023-10-12 20:40:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35356672. Throughput: 0: 1636.9, 1: 1645.1. Samples: 8843366. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:40:51,443][43579] Avg episode reward: [(0, '266.200'), (1, '261.480')] [2023-10-12 20:40:52,723][44958] Updated weights for policy 0, policy_version 17220 (0.0007) [2023-10-12 20:40:53,089][44958] Updated weights for policy 0, policy_version 17230 (0.0008) [2023-10-12 20:40:53,275][44959] Updated weights for policy 1, policy_version 17320 (0.0008) [2023-10-12 20:40:53,471][44958] Updated weights for policy 0, policy_version 17240 (0.0008) [2023-10-12 20:40:53,646][44959] Updated weights for policy 1, policy_version 17330 (0.0009) [2023-10-12 20:40:54,014][44959] Updated weights for policy 1, policy_version 17340 (0.0009) [2023-10-12 20:40:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35422208. Throughput: 0: 1646.5, 1: 1655.2. Samples: 8864034. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 20:40:56,443][43579] Avg episode reward: [(0, '256.830'), (1, '265.700')] [2023-10-12 20:40:57,550][44958] Updated weights for policy 0, policy_version 17250 (0.0008) [2023-10-12 20:40:57,930][44958] Updated weights for policy 0, policy_version 17260 (0.0008) [2023-10-12 20:40:58,073][44959] Updated weights for policy 1, policy_version 17350 (0.0008) [2023-10-12 20:40:58,299][44958] Updated weights for policy 0, policy_version 17270 (0.0009) [2023-10-12 20:40:58,446][44959] Updated weights for policy 1, policy_version 17360 (0.0007) [2023-10-12 20:40:58,666][44958] Updated weights for policy 0, policy_version 17280 (0.0008) [2023-10-12 20:40:58,806][44959] Updated weights for policy 1, policy_version 17370 (0.0009) [2023-10-12 20:41:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35487744. Throughput: 0: 1648.5, 1: 1652.0. Samples: 8884304. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 20:41:01,443][43579] Avg episode reward: [(0, '255.690'), (1, '270.740')] [2023-10-12 20:41:01,452][44583] Saving new best policy, reward=270.740! [2023-10-12 20:41:02,769][44958] Updated weights for policy 0, policy_version 17290 (0.0009) [2023-10-12 20:41:03,052][44959] Updated weights for policy 1, policy_version 17380 (0.0009) [2023-10-12 20:41:03,147][44958] Updated weights for policy 0, policy_version 17300 (0.0009) [2023-10-12 20:41:03,449][44959] Updated weights for policy 1, policy_version 17390 (0.0009) [2023-10-12 20:41:03,517][44958] Updated weights for policy 0, policy_version 17310 (0.0009) [2023-10-12 20:41:03,820][44959] Updated weights for policy 1, policy_version 17400 (0.0010) [2023-10-12 20:41:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35553280. Throughput: 0: 1645.7, 1: 1646.0. Samples: 8893238. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 20:41:06,444][43579] Avg episode reward: [(0, '259.930'), (1, '273.490')] [2023-10-12 20:41:06,445][44583] Saving new best policy, reward=273.490! [2023-10-12 20:41:07,692][44958] Updated weights for policy 0, policy_version 17320 (0.0007) [2023-10-12 20:41:07,823][44959] Updated weights for policy 1, policy_version 17410 (0.0009) [2023-10-12 20:41:08,073][44958] Updated weights for policy 0, policy_version 17330 (0.0008) [2023-10-12 20:41:08,191][44959] Updated weights for policy 1, policy_version 17420 (0.0007) [2023-10-12 20:41:08,440][44958] Updated weights for policy 0, policy_version 17340 (0.0009) [2023-10-12 20:41:08,555][44959] Updated weights for policy 1, policy_version 17430 (0.0007) [2023-10-12 20:41:08,920][44959] Updated weights for policy 1, policy_version 17440 (0.0008) [2023-10-12 20:41:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35618816. Throughput: 0: 1642.2, 1: 1659.5. Samples: 8913630. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 20:41:11,443][43579] Avg episode reward: [(0, '253.130'), (1, '276.170')] [2023-10-12 20:41:11,444][44583] Saving new best policy, reward=276.170! [2023-10-12 20:41:12,656][44958] Updated weights for policy 0, policy_version 17350 (0.0008) [2023-10-12 20:41:13,033][44958] Updated weights for policy 0, policy_version 17360 (0.0007) [2023-10-12 20:41:13,152][44959] Updated weights for policy 1, policy_version 17450 (0.0007) [2023-10-12 20:41:13,399][44958] Updated weights for policy 0, policy_version 17370 (0.0009) [2023-10-12 20:41:13,512][44959] Updated weights for policy 1, policy_version 17460 (0.0008) [2023-10-12 20:41:13,885][44959] Updated weights for policy 1, policy_version 17470 (0.0009) [2023-10-12 20:41:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35684352. Throughput: 0: 1650.1, 1: 1656.2. Samples: 8933856. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 20:41:16,444][43579] Avg episode reward: [(0, '254.810'), (1, '273.880')] [2023-10-12 20:41:17,822][44958] Updated weights for policy 0, policy_version 17380 (0.0010) [2023-10-12 20:41:18,007][44959] Updated weights for policy 1, policy_version 17480 (0.0007) [2023-10-12 20:41:18,198][44958] Updated weights for policy 0, policy_version 17390 (0.0010) [2023-10-12 20:41:18,376][44959] Updated weights for policy 1, policy_version 17490 (0.0010) [2023-10-12 20:41:18,566][44958] Updated weights for policy 0, policy_version 17400 (0.0008) [2023-10-12 20:41:18,750][44959] Updated weights for policy 1, policy_version 17500 (0.0009) [2023-10-12 20:41:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 35749888. Throughput: 0: 1650.8, 1: 1646.1. Samples: 8942598. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 20:41:21,444][43579] Avg episode reward: [(0, '249.680'), (1, '272.670')] [2023-10-12 20:41:22,728][44958] Updated weights for policy 0, policy_version 17410 (0.0008) [2023-10-12 20:41:22,884][44959] Updated weights for policy 1, policy_version 17510 (0.0007) [2023-10-12 20:41:23,098][44958] Updated weights for policy 0, policy_version 17420 (0.0009) [2023-10-12 20:41:23,249][44959] Updated weights for policy 1, policy_version 17520 (0.0009) [2023-10-12 20:41:23,474][44958] Updated weights for policy 0, policy_version 17430 (0.0010) [2023-10-12 20:41:23,628][44959] Updated weights for policy 1, policy_version 17530 (0.0007) [2023-10-12 20:41:23,852][44958] Updated weights for policy 0, policy_version 17440 (0.0008) [2023-10-12 20:41:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35815424. Throughput: 0: 1646.3, 1: 1660.1. Samples: 8962852. Policy #0 lag: (min: 31.0, avg: 32.7, max: 60.0) [2023-10-12 20:41:26,443][43579] Avg episode reward: [(0, '252.160'), (1, '270.530')] [2023-10-12 20:41:27,899][44959] Updated weights for policy 1, policy_version 17540 (0.0008) [2023-10-12 20:41:28,034][44958] Updated weights for policy 0, policy_version 17450 (0.0007) [2023-10-12 20:41:28,268][44959] Updated weights for policy 1, policy_version 17550 (0.0009) [2023-10-12 20:41:28,395][44958] Updated weights for policy 0, policy_version 17460 (0.0007) [2023-10-12 20:41:28,635][44959] Updated weights for policy 1, policy_version 17560 (0.0008) [2023-10-12 20:41:28,764][44958] Updated weights for policy 0, policy_version 17470 (0.0008) [2023-10-12 20:41:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35880960. Throughput: 0: 1645.1, 1: 1658.7. Samples: 8982966. Policy #0 lag: (min: 31.0, avg: 32.7, max: 60.0) [2023-10-12 20:41:31,444][43579] Avg episode reward: [(0, '253.940'), (1, '271.380')] [2023-10-12 20:41:32,639][44959] Updated weights for policy 1, policy_version 17570 (0.0010) [2023-10-12 20:41:32,995][44959] Updated weights for policy 1, policy_version 17580 (0.0009) [2023-10-12 20:41:33,037][44958] Updated weights for policy 0, policy_version 17480 (0.0007) [2023-10-12 20:41:33,366][44959] Updated weights for policy 1, policy_version 17590 (0.0009) [2023-10-12 20:41:33,413][44958] Updated weights for policy 0, policy_version 17490 (0.0009) [2023-10-12 20:41:33,731][44959] Updated weights for policy 1, policy_version 17600 (0.0008) [2023-10-12 20:41:33,783][44958] Updated weights for policy 0, policy_version 17500 (0.0008) [2023-10-12 20:41:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 35946496. Throughput: 0: 1648.4, 1: 1647.9. Samples: 8991700. Policy #0 lag: (min: 31.0, avg: 32.7, max: 60.0) [2023-10-12 20:41:36,443][43579] Avg episode reward: [(0, '264.750'), (1, '265.470')] [2023-10-12 20:41:37,855][44959] Updated weights for policy 1, policy_version 17610 (0.0009) [2023-10-12 20:41:37,959][44958] Updated weights for policy 0, policy_version 17510 (0.0011) [2023-10-12 20:41:38,215][44959] Updated weights for policy 1, policy_version 17620 (0.0007) [2023-10-12 20:41:38,317][44958] Updated weights for policy 0, policy_version 17520 (0.0009) [2023-10-12 20:41:38,583][44959] Updated weights for policy 1, policy_version 17630 (0.0008) [2023-10-12 20:41:38,690][44958] Updated weights for policy 0, policy_version 17530 (0.0010) [2023-10-12 20:41:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36012032. Throughput: 0: 1631.7, 1: 1651.6. Samples: 9011782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:41:41,443][43579] Avg episode reward: [(0, '269.780'), (1, '256.340')] [2023-10-12 20:41:42,637][44959] Updated weights for policy 1, policy_version 17640 (0.0008) [2023-10-12 20:41:42,889][44958] Updated weights for policy 0, policy_version 17540 (0.0009) [2023-10-12 20:41:43,011][44959] Updated weights for policy 1, policy_version 17650 (0.0009) [2023-10-12 20:41:43,252][44958] Updated weights for policy 0, policy_version 17550 (0.0008) [2023-10-12 20:41:43,374][44959] Updated weights for policy 1, policy_version 17660 (0.0010) [2023-10-12 20:41:43,634][44958] Updated weights for policy 0, policy_version 17560 (0.0009) [2023-10-12 20:41:46,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 36077568. Throughput: 0: 1631.3, 1: 1659.2. Samples: 9032380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:41:46,444][43579] Avg episode reward: [(0, '266.340'), (1, '259.500')] [2023-10-12 20:41:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000017664_18087936.pth... [2023-10-12 20:41:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000017568_17989632.pth... [2023-10-12 20:41:46,490][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000016032_16416768.pth [2023-10-12 20:41:46,496][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000016128_16515072.pth [2023-10-12 20:41:47,529][44959] Updated weights for policy 1, policy_version 17670 (0.0008) [2023-10-12 20:41:47,846][44958] Updated weights for policy 0, policy_version 17570 (0.0008) [2023-10-12 20:41:47,895][44959] Updated weights for policy 1, policy_version 17680 (0.0008) [2023-10-12 20:41:48,205][44958] Updated weights for policy 0, policy_version 17580 (0.0008) [2023-10-12 20:41:48,257][44959] Updated weights for policy 1, policy_version 17690 (0.0009) [2023-10-12 20:41:48,582][44958] Updated weights for policy 0, policy_version 17590 (0.0008) [2023-10-12 20:41:48,943][44958] Updated weights for policy 0, policy_version 17600 (0.0008) [2023-10-12 20:41:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36143104. Throughput: 0: 1632.4, 1: 1655.2. Samples: 9041184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:41:51,443][43579] Avg episode reward: [(0, '271.230'), (1, '259.170')] [2023-10-12 20:41:52,468][44959] Updated weights for policy 1, policy_version 17700 (0.0009) [2023-10-12 20:41:52,831][44959] Updated weights for policy 1, policy_version 17710 (0.0007) [2023-10-12 20:41:52,882][44958] Updated weights for policy 0, policy_version 17610 (0.0008) [2023-10-12 20:41:53,207][44959] Updated weights for policy 1, policy_version 17720 (0.0009) [2023-10-12 20:41:53,259][44958] Updated weights for policy 0, policy_version 17620 (0.0008) [2023-10-12 20:41:53,625][44958] Updated weights for policy 0, policy_version 17630 (0.0008) [2023-10-12 20:41:56,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36208640. Throughput: 0: 1631.2, 1: 1648.7. Samples: 9061228. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-12 20:41:56,443][43579] Avg episode reward: [(0, '267.490'), (1, '260.820')] [2023-10-12 20:41:57,419][44959] Updated weights for policy 1, policy_version 17730 (0.0009) [2023-10-12 20:41:57,779][44959] Updated weights for policy 1, policy_version 17740 (0.0008) [2023-10-12 20:41:57,915][44958] Updated weights for policy 0, policy_version 17640 (0.0007) [2023-10-12 20:41:58,140][44959] Updated weights for policy 1, policy_version 17750 (0.0008) [2023-10-12 20:41:58,288][44958] Updated weights for policy 0, policy_version 17650 (0.0008) [2023-10-12 20:41:58,510][44959] Updated weights for policy 1, policy_version 17760 (0.0008) [2023-10-12 20:41:58,663][44958] Updated weights for policy 0, policy_version 17660 (0.0007) [2023-10-12 20:42:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36274176. Throughput: 0: 1628.4, 1: 1648.0. Samples: 9081298. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-12 20:42:01,444][43579] Avg episode reward: [(0, '268.990'), (1, '257.610')] [2023-10-12 20:42:02,811][44959] Updated weights for policy 1, policy_version 17770 (0.0009) [2023-10-12 20:42:02,938][44958] Updated weights for policy 0, policy_version 17670 (0.0008) [2023-10-12 20:42:03,179][44959] Updated weights for policy 1, policy_version 17780 (0.0008) [2023-10-12 20:42:03,305][44958] Updated weights for policy 0, policy_version 17680 (0.0007) [2023-10-12 20:42:03,540][44959] Updated weights for policy 1, policy_version 17790 (0.0009) [2023-10-12 20:42:03,672][44958] Updated weights for policy 0, policy_version 17690 (0.0009) [2023-10-12 20:42:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36339712. Throughput: 0: 1626.5, 1: 1649.3. Samples: 9090010. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-12 20:42:06,443][43579] Avg episode reward: [(0, '266.690'), (1, '264.360')] [2023-10-12 20:42:07,828][44958] Updated weights for policy 0, policy_version 17700 (0.0009) [2023-10-12 20:42:07,875][44959] Updated weights for policy 1, policy_version 17800 (0.0008) [2023-10-12 20:42:08,205][44958] Updated weights for policy 0, policy_version 17710 (0.0008) [2023-10-12 20:42:08,254][44959] Updated weights for policy 1, policy_version 17810 (0.0008) [2023-10-12 20:42:08,576][44958] Updated weights for policy 0, policy_version 17720 (0.0007) [2023-10-12 20:42:08,621][44959] Updated weights for policy 1, policy_version 17820 (0.0008) [2023-10-12 20:42:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36405248. Throughput: 0: 1632.1, 1: 1643.4. Samples: 9110248. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 20:42:11,443][43579] Avg episode reward: [(0, '264.490'), (1, '264.940')] [2023-10-12 20:42:12,743][44959] Updated weights for policy 1, policy_version 17830 (0.0008) [2023-10-12 20:42:12,760][44958] Updated weights for policy 0, policy_version 17730 (0.0008) [2023-10-12 20:42:13,107][44959] Updated weights for policy 1, policy_version 17840 (0.0010) [2023-10-12 20:42:13,125][44958] Updated weights for policy 0, policy_version 17740 (0.0007) [2023-10-12 20:42:13,469][44959] Updated weights for policy 1, policy_version 17850 (0.0010) [2023-10-12 20:42:13,497][44958] Updated weights for policy 0, policy_version 17750 (0.0007) [2023-10-12 20:42:13,871][44958] Updated weights for policy 0, policy_version 17760 (0.0011) [2023-10-12 20:42:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36470784. Throughput: 0: 1640.5, 1: 1639.9. Samples: 9130584. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 20:42:16,444][43579] Avg episode reward: [(0, '267.960'), (1, '266.760')] [2023-10-12 20:42:17,546][44959] Updated weights for policy 1, policy_version 17860 (0.0008) [2023-10-12 20:42:17,913][44959] Updated weights for policy 1, policy_version 17870 (0.0008) [2023-10-12 20:42:17,944][44958] Updated weights for policy 0, policy_version 17770 (0.0007) [2023-10-12 20:42:18,273][44959] Updated weights for policy 1, policy_version 17880 (0.0010) [2023-10-12 20:42:18,319][44958] Updated weights for policy 0, policy_version 17780 (0.0007) [2023-10-12 20:42:18,696][44958] Updated weights for policy 0, policy_version 17790 (0.0010) [2023-10-12 20:42:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36536320. Throughput: 0: 1637.4, 1: 1641.9. Samples: 9139270. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 20:42:21,444][43579] Avg episode reward: [(0, '269.510'), (1, '260.170')] [2023-10-12 20:42:22,582][44959] Updated weights for policy 1, policy_version 17890 (0.0007) [2023-10-12 20:42:22,902][44958] Updated weights for policy 0, policy_version 17800 (0.0009) [2023-10-12 20:42:22,943][44959] Updated weights for policy 1, policy_version 17900 (0.0007) [2023-10-12 20:42:23,269][44958] Updated weights for policy 0, policy_version 17810 (0.0009) [2023-10-12 20:42:23,315][44959] Updated weights for policy 1, policy_version 17910 (0.0007) [2023-10-12 20:42:23,650][44958] Updated weights for policy 0, policy_version 17820 (0.0010) [2023-10-12 20:42:23,687][44959] Updated weights for policy 1, policy_version 17920 (0.0009) [2023-10-12 20:42:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36601856. Throughput: 0: 1640.6, 1: 1642.5. Samples: 9159524. Policy #0 lag: (min: 18.0, avg: 24.8, max: 50.0) [2023-10-12 20:42:26,444][43579] Avg episode reward: [(0, '265.520'), (1, '260.800')] [2023-10-12 20:42:27,906][44959] Updated weights for policy 1, policy_version 17930 (0.0008) [2023-10-12 20:42:27,947][44958] Updated weights for policy 0, policy_version 17830 (0.0009) [2023-10-12 20:42:28,264][44959] Updated weights for policy 1, policy_version 17940 (0.0008) [2023-10-12 20:42:28,314][44958] Updated weights for policy 0, policy_version 17840 (0.0009) [2023-10-12 20:42:28,631][44959] Updated weights for policy 1, policy_version 17950 (0.0007) [2023-10-12 20:42:28,683][44958] Updated weights for policy 0, policy_version 17850 (0.0008) [2023-10-12 20:42:31,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 36667392. Throughput: 0: 1637.7, 1: 1636.8. Samples: 9179732. Policy #0 lag: (min: 18.0, avg: 24.8, max: 50.0) [2023-10-12 20:42:31,443][43579] Avg episode reward: [(0, '261.960'), (1, '253.160')] [2023-10-12 20:42:32,822][44958] Updated weights for policy 0, policy_version 17860 (0.0009) [2023-10-12 20:42:32,945][44959] Updated weights for policy 1, policy_version 17960 (0.0007) [2023-10-12 20:42:33,196][44958] Updated weights for policy 0, policy_version 17870 (0.0007) [2023-10-12 20:42:33,315][44959] Updated weights for policy 1, policy_version 17970 (0.0008) [2023-10-12 20:42:33,564][44958] Updated weights for policy 0, policy_version 17880 (0.0008) [2023-10-12 20:42:33,679][44959] Updated weights for policy 1, policy_version 17980 (0.0009) [2023-10-12 20:42:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36732928. Throughput: 0: 1637.7, 1: 1634.5. Samples: 9188434. Policy #0 lag: (min: 18.0, avg: 24.8, max: 50.0) [2023-10-12 20:42:36,443][43579] Avg episode reward: [(0, '259.210'), (1, '259.710')] [2023-10-12 20:42:37,604][44958] Updated weights for policy 0, policy_version 17890 (0.0010) [2023-10-12 20:42:37,728][44959] Updated weights for policy 1, policy_version 17990 (0.0007) [2023-10-12 20:42:37,972][44958] Updated weights for policy 0, policy_version 17900 (0.0010) [2023-10-12 20:42:38,094][44959] Updated weights for policy 1, policy_version 18000 (0.0008) [2023-10-12 20:42:38,344][44958] Updated weights for policy 0, policy_version 17910 (0.0009) [2023-10-12 20:42:38,464][44959] Updated weights for policy 1, policy_version 18010 (0.0010) [2023-10-12 20:42:38,725][44958] Updated weights for policy 0, policy_version 17920 (0.0009) [2023-10-12 20:42:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36798464. Throughput: 0: 1638.7, 1: 1644.0. Samples: 9208948. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-12 20:42:41,443][43579] Avg episode reward: [(0, '259.460'), (1, '258.230')] [2023-10-12 20:42:42,561][44959] Updated weights for policy 1, policy_version 18020 (0.0008) [2023-10-12 20:42:42,926][44959] Updated weights for policy 1, policy_version 18030 (0.0008) [2023-10-12 20:42:42,936][44958] Updated weights for policy 0, policy_version 17930 (0.0007) [2023-10-12 20:42:43,290][44959] Updated weights for policy 1, policy_version 18040 (0.0007) [2023-10-12 20:42:43,313][44958] Updated weights for policy 0, policy_version 17940 (0.0007) [2023-10-12 20:42:43,682][44958] Updated weights for policy 0, policy_version 17950 (0.0009) [2023-10-12 20:42:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 36864000. Throughput: 0: 1641.3, 1: 1644.9. Samples: 9229176. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-12 20:42:46,443][43579] Avg episode reward: [(0, '257.220'), (1, '259.290')] [2023-10-12 20:42:47,483][44959] Updated weights for policy 1, policy_version 18050 (0.0008) [2023-10-12 20:42:47,856][44959] Updated weights for policy 1, policy_version 18060 (0.0007) [2023-10-12 20:42:47,913][44958] Updated weights for policy 0, policy_version 17960 (0.0007) [2023-10-12 20:42:48,232][44959] Updated weights for policy 1, policy_version 18070 (0.0008) [2023-10-12 20:42:48,290][44958] Updated weights for policy 0, policy_version 17970 (0.0008) [2023-10-12 20:42:48,593][44959] Updated weights for policy 1, policy_version 18080 (0.0009) [2023-10-12 20:42:48,659][44958] Updated weights for policy 0, policy_version 17980 (0.0009) [2023-10-12 20:42:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36929536. Throughput: 0: 1644.3, 1: 1645.0. Samples: 9238026. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-12 20:42:51,443][43579] Avg episode reward: [(0, '255.900'), (1, '261.420')] [2023-10-12 20:42:52,655][44958] Updated weights for policy 0, policy_version 17990 (0.0008) [2023-10-12 20:42:52,925][44959] Updated weights for policy 1, policy_version 18090 (0.0009) [2023-10-12 20:42:53,015][44958] Updated weights for policy 0, policy_version 18000 (0.0008) [2023-10-12 20:42:53,287][44959] Updated weights for policy 1, policy_version 18100 (0.0009) [2023-10-12 20:42:53,398][44958] Updated weights for policy 0, policy_version 18010 (0.0009) [2023-10-12 20:42:53,652][44959] Updated weights for policy 1, policy_version 18110 (0.0008) [2023-10-12 20:42:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 36995072. Throughput: 0: 1641.7, 1: 1644.4. Samples: 9258122. Policy #0 lag: (min: 26.0, avg: 53.4, max: 56.0) [2023-10-12 20:42:56,444][43579] Avg episode reward: [(0, '257.980'), (1, '255.540')] [2023-10-12 20:42:57,539][44958] Updated weights for policy 0, policy_version 18020 (0.0008) [2023-10-12 20:42:57,914][44958] Updated weights for policy 0, policy_version 18030 (0.0007) [2023-10-12 20:42:57,919][44959] Updated weights for policy 1, policy_version 18120 (0.0009) [2023-10-12 20:42:58,277][44959] Updated weights for policy 1, policy_version 18130 (0.0008) [2023-10-12 20:42:58,283][44958] Updated weights for policy 0, policy_version 18040 (0.0007) [2023-10-12 20:42:58,646][44959] Updated weights for policy 1, policy_version 18140 (0.0009) [2023-10-12 20:43:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37060608. Throughput: 0: 1635.5, 1: 1646.4. Samples: 9278268. Policy #0 lag: (min: 26.0, avg: 53.4, max: 56.0) [2023-10-12 20:43:01,444][43579] Avg episode reward: [(0, '266.310'), (1, '254.280')] [2023-10-12 20:43:02,595][44959] Updated weights for policy 1, policy_version 18150 (0.0007) [2023-10-12 20:43:02,819][44958] Updated weights for policy 0, policy_version 18050 (0.0007) [2023-10-12 20:43:02,961][44959] Updated weights for policy 1, policy_version 18160 (0.0007) [2023-10-12 20:43:03,182][44958] Updated weights for policy 0, policy_version 18060 (0.0007) [2023-10-12 20:43:03,323][44959] Updated weights for policy 1, policy_version 18170 (0.0009) [2023-10-12 20:43:03,555][44958] Updated weights for policy 0, policy_version 18070 (0.0008) [2023-10-12 20:43:03,928][44958] Updated weights for policy 0, policy_version 18080 (0.0011) [2023-10-12 20:43:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37126144. Throughput: 0: 1637.4, 1: 1650.0. Samples: 9287202. Policy #0 lag: (min: 26.0, avg: 53.4, max: 56.0) [2023-10-12 20:43:06,444][43579] Avg episode reward: [(0, '263.660'), (1, '249.290')] [2023-10-12 20:43:07,560][44959] Updated weights for policy 1, policy_version 18180 (0.0008) [2023-10-12 20:43:07,928][44959] Updated weights for policy 1, policy_version 18190 (0.0007) [2023-10-12 20:43:07,945][44958] Updated weights for policy 0, policy_version 18090 (0.0008) [2023-10-12 20:43:08,289][44959] Updated weights for policy 1, policy_version 18200 (0.0008) [2023-10-12 20:43:08,321][44958] Updated weights for policy 0, policy_version 18100 (0.0008) [2023-10-12 20:43:08,685][44958] Updated weights for policy 0, policy_version 18110 (0.0009) [2023-10-12 20:43:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37191680. Throughput: 0: 1640.2, 1: 1646.7. Samples: 9307432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:43:11,444][43579] Avg episode reward: [(0, '263.880'), (1, '251.040')] [2023-10-12 20:43:12,449][44959] Updated weights for policy 1, policy_version 18210 (0.0009) [2023-10-12 20:43:12,815][44959] Updated weights for policy 1, policy_version 18220 (0.0007) [2023-10-12 20:43:12,937][44958] Updated weights for policy 0, policy_version 18120 (0.0010) [2023-10-12 20:43:13,184][44959] Updated weights for policy 1, policy_version 18230 (0.0008) [2023-10-12 20:43:13,313][44958] Updated weights for policy 0, policy_version 18130 (0.0008) [2023-10-12 20:43:13,544][44959] Updated weights for policy 1, policy_version 18240 (0.0008) [2023-10-12 20:43:13,683][44958] Updated weights for policy 0, policy_version 18140 (0.0009) [2023-10-12 20:43:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37257216. Throughput: 0: 1638.6, 1: 1651.4. Samples: 9327784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:43:16,444][43579] Avg episode reward: [(0, '263.000'), (1, '253.060')] [2023-10-12 20:43:17,706][44959] Updated weights for policy 1, policy_version 18250 (0.0009) [2023-10-12 20:43:17,888][44958] Updated weights for policy 0, policy_version 18150 (0.0007) [2023-10-12 20:43:18,080][44959] Updated weights for policy 1, policy_version 18260 (0.0008) [2023-10-12 20:43:18,264][44958] Updated weights for policy 0, policy_version 18160 (0.0007) [2023-10-12 20:43:18,444][44959] Updated weights for policy 1, policy_version 18270 (0.0009) [2023-10-12 20:43:18,635][44958] Updated weights for policy 0, policy_version 18170 (0.0009) [2023-10-12 20:43:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 37322752. Throughput: 0: 1637.4, 1: 1650.6. Samples: 9336394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:43:21,443][43579] Avg episode reward: [(0, '258.770'), (1, '257.360')] [2023-10-12 20:43:22,704][44959] Updated weights for policy 1, policy_version 18280 (0.0008) [2023-10-12 20:43:22,846][44958] Updated weights for policy 0, policy_version 18180 (0.0008) [2023-10-12 20:43:23,067][44959] Updated weights for policy 1, policy_version 18290 (0.0007) [2023-10-12 20:43:23,216][44958] Updated weights for policy 0, policy_version 18190 (0.0009) [2023-10-12 20:43:23,431][44959] Updated weights for policy 1, policy_version 18300 (0.0008) [2023-10-12 20:43:23,598][44958] Updated weights for policy 0, policy_version 18200 (0.0009) [2023-10-12 20:43:26,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37388288. Throughput: 0: 1636.8, 1: 1642.0. Samples: 9356494. Policy #0 lag: (min: 16.0, avg: 37.6, max: 48.0) [2023-10-12 20:43:26,444][43579] Avg episode reward: [(0, '257.220'), (1, '263.700')] [2023-10-12 20:43:27,590][44959] Updated weights for policy 1, policy_version 18310 (0.0008) [2023-10-12 20:43:27,888][44958] Updated weights for policy 0, policy_version 18210 (0.0010) [2023-10-12 20:43:27,965][44959] Updated weights for policy 1, policy_version 18320 (0.0008) [2023-10-12 20:43:28,308][44958] Updated weights for policy 0, policy_version 18220 (0.0008) [2023-10-12 20:43:28,334][44959] Updated weights for policy 1, policy_version 18330 (0.0009) [2023-10-12 20:43:28,676][44958] Updated weights for policy 0, policy_version 18230 (0.0010) [2023-10-12 20:43:29,043][44958] Updated weights for policy 0, policy_version 18240 (0.0009) [2023-10-12 20:43:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 37453824. Throughput: 0: 1634.7, 1: 1644.2. Samples: 9376726. Policy #0 lag: (min: 16.0, avg: 37.6, max: 48.0) [2023-10-12 20:43:31,444][43579] Avg episode reward: [(0, '255.960'), (1, '265.920')] [2023-10-12 20:43:32,310][44959] Updated weights for policy 1, policy_version 18340 (0.0010) [2023-10-12 20:43:32,669][44959] Updated weights for policy 1, policy_version 18350 (0.0010) [2023-10-12 20:43:33,036][44959] Updated weights for policy 1, policy_version 18360 (0.0008) [2023-10-12 20:43:33,172][44958] Updated weights for policy 0, policy_version 18250 (0.0009) [2023-10-12 20:43:33,540][44958] Updated weights for policy 0, policy_version 18260 (0.0009) [2023-10-12 20:43:33,924][44958] Updated weights for policy 0, policy_version 18270 (0.0009) [2023-10-12 20:43:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37519360. Throughput: 0: 1632.5, 1: 1644.3. Samples: 9385484. Policy #0 lag: (min: 16.0, avg: 37.6, max: 48.0) [2023-10-12 20:43:36,444][43579] Avg episode reward: [(0, '252.970'), (1, '273.980')] [2023-10-12 20:43:37,352][44959] Updated weights for policy 1, policy_version 18370 (0.0011) [2023-10-12 20:43:37,727][44959] Updated weights for policy 1, policy_version 18380 (0.0008) [2023-10-12 20:43:38,040][44958] Updated weights for policy 0, policy_version 18280 (0.0008) [2023-10-12 20:43:38,091][44959] Updated weights for policy 1, policy_version 18390 (0.0007) [2023-10-12 20:43:38,408][44958] Updated weights for policy 0, policy_version 18290 (0.0008) [2023-10-12 20:43:38,463][44959] Updated weights for policy 1, policy_version 18400 (0.0007) [2023-10-12 20:43:38,789][44958] Updated weights for policy 0, policy_version 18300 (0.0010) [2023-10-12 20:43:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37584896. Throughput: 0: 1634.5, 1: 1646.8. Samples: 9405782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:43:41,444][43579] Avg episode reward: [(0, '250.340'), (1, '275.580')] [2023-10-12 20:43:42,463][44959] Updated weights for policy 1, policy_version 18410 (0.0009) [2023-10-12 20:43:42,838][44959] Updated weights for policy 1, policy_version 18420 (0.0007) [2023-10-12 20:43:42,860][44958] Updated weights for policy 0, policy_version 18310 (0.0008) [2023-10-12 20:43:43,206][44959] Updated weights for policy 1, policy_version 18430 (0.0009) [2023-10-12 20:43:43,230][44958] Updated weights for policy 0, policy_version 18320 (0.0009) [2023-10-12 20:43:43,606][44958] Updated weights for policy 0, policy_version 18330 (0.0009) [2023-10-12 20:43:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37650432. Throughput: 0: 1632.8, 1: 1651.5. Samples: 9426060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:43:46,444][43579] Avg episode reward: [(0, '259.240'), (1, '273.040')] [2023-10-12 20:43:46,457][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000018336_18776064.pth... [2023-10-12 20:43:46,457][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000018432_18874368.pth... [2023-10-12 20:43:46,492][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000016896_17301504.pth [2023-10-12 20:43:46,493][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000016800_17203200.pth [2023-10-12 20:43:47,351][44959] Updated weights for policy 1, policy_version 18440 (0.0010) [2023-10-12 20:43:47,720][44959] Updated weights for policy 1, policy_version 18450 (0.0009) [2023-10-12 20:43:47,911][44958] Updated weights for policy 0, policy_version 18340 (0.0009) [2023-10-12 20:43:48,098][44959] Updated weights for policy 1, policy_version 18460 (0.0009) [2023-10-12 20:43:48,283][44958] Updated weights for policy 0, policy_version 18350 (0.0009) [2023-10-12 20:43:48,651][44958] Updated weights for policy 0, policy_version 18360 (0.0007) [2023-10-12 20:43:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 37715968. Throughput: 0: 1633.7, 1: 1646.7. Samples: 9434820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:43:51,444][43579] Avg episode reward: [(0, '265.470'), (1, '277.120')] [2023-10-12 20:43:51,445][44583] Saving new best policy, reward=277.120! [2023-10-12 20:43:52,405][44959] Updated weights for policy 1, policy_version 18470 (0.0007) [2023-10-12 20:43:52,770][44959] Updated weights for policy 1, policy_version 18480 (0.0007) [2023-10-12 20:43:52,809][44958] Updated weights for policy 0, policy_version 18370 (0.0009) [2023-10-12 20:43:53,130][44959] Updated weights for policy 1, policy_version 18490 (0.0008) [2023-10-12 20:43:53,176][44958] Updated weights for policy 0, policy_version 18380 (0.0007) [2023-10-12 20:43:53,556][44958] Updated weights for policy 0, policy_version 18390 (0.0007) [2023-10-12 20:43:53,924][44958] Updated weights for policy 0, policy_version 18400 (0.0007) [2023-10-12 20:43:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37781504. Throughput: 0: 1634.6, 1: 1649.3. Samples: 9455206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:43:56,444][43579] Avg episode reward: [(0, '266.330'), (1, '276.550')] [2023-10-12 20:43:57,323][44959] Updated weights for policy 1, policy_version 18500 (0.0009) [2023-10-12 20:43:57,690][44959] Updated weights for policy 1, policy_version 18510 (0.0010) [2023-10-12 20:43:58,067][44959] Updated weights for policy 1, policy_version 18520 (0.0007) [2023-10-12 20:43:58,141][44958] Updated weights for policy 0, policy_version 18410 (0.0008) [2023-10-12 20:43:58,509][44958] Updated weights for policy 0, policy_version 18420 (0.0007) [2023-10-12 20:43:58,874][44958] Updated weights for policy 0, policy_version 18430 (0.0008) [2023-10-12 20:44:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37847040. Throughput: 0: 1631.7, 1: 1642.8. Samples: 9475134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:44:01,443][43579] Avg episode reward: [(0, '265.200'), (1, '280.710')] [2023-10-12 20:44:01,451][44583] Saving new best policy, reward=280.710! [2023-10-12 20:44:02,227][44959] Updated weights for policy 1, policy_version 18530 (0.0010) [2023-10-12 20:44:02,601][44959] Updated weights for policy 1, policy_version 18540 (0.0009) [2023-10-12 20:44:02,964][44959] Updated weights for policy 1, policy_version 18550 (0.0008) [2023-10-12 20:44:03,197][44958] Updated weights for policy 0, policy_version 18440 (0.0009) [2023-10-12 20:44:03,325][44959] Updated weights for policy 1, policy_version 18560 (0.0008) [2023-10-12 20:44:03,568][44958] Updated weights for policy 0, policy_version 18450 (0.0010) [2023-10-12 20:44:03,935][44958] Updated weights for policy 0, policy_version 18460 (0.0009) [2023-10-12 20:44:06,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 37912576. Throughput: 0: 1632.7, 1: 1645.8. Samples: 9483930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:44:06,444][43579] Avg episode reward: [(0, '260.150'), (1, '274.930')] [2023-10-12 20:44:07,651][44959] Updated weights for policy 1, policy_version 18570 (0.0008) [2023-10-12 20:44:08,027][44959] Updated weights for policy 1, policy_version 18580 (0.0007) [2023-10-12 20:44:08,081][44958] Updated weights for policy 0, policy_version 18470 (0.0008) [2023-10-12 20:44:08,397][44959] Updated weights for policy 1, policy_version 18590 (0.0007) [2023-10-12 20:44:08,446][44958] Updated weights for policy 0, policy_version 18480 (0.0008) [2023-10-12 20:44:08,819][44958] Updated weights for policy 0, policy_version 18490 (0.0008) [2023-10-12 20:44:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 37978112. Throughput: 0: 1632.8, 1: 1642.1. Samples: 9503864. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:44:11,443][43579] Avg episode reward: [(0, '262.540'), (1, '272.800')] [2023-10-12 20:44:12,507][44959] Updated weights for policy 1, policy_version 18600 (0.0007) [2023-10-12 20:44:12,868][44959] Updated weights for policy 1, policy_version 18610 (0.0008) [2023-10-12 20:44:13,064][44958] Updated weights for policy 0, policy_version 18500 (0.0009) [2023-10-12 20:44:13,241][44959] Updated weights for policy 1, policy_version 18620 (0.0007) [2023-10-12 20:44:13,452][44958] Updated weights for policy 0, policy_version 18510 (0.0010) [2023-10-12 20:44:13,819][44958] Updated weights for policy 0, policy_version 18520 (0.0008) [2023-10-12 20:44:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38043648. Throughput: 0: 1636.5, 1: 1644.3. Samples: 9524362. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:44:16,443][43579] Avg episode reward: [(0, '258.120'), (1, '270.680')] [2023-10-12 20:44:17,333][44959] Updated weights for policy 1, policy_version 18630 (0.0010) [2023-10-12 20:44:17,711][44959] Updated weights for policy 1, policy_version 18640 (0.0008) [2023-10-12 20:44:17,916][44958] Updated weights for policy 0, policy_version 18530 (0.0007) [2023-10-12 20:44:18,077][44959] Updated weights for policy 1, policy_version 18650 (0.0010) [2023-10-12 20:44:18,281][44958] Updated weights for policy 0, policy_version 18540 (0.0008) [2023-10-12 20:44:18,646][44958] Updated weights for policy 0, policy_version 18550 (0.0010) [2023-10-12 20:44:19,014][44958] Updated weights for policy 0, policy_version 18560 (0.0009) [2023-10-12 20:44:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38109184. Throughput: 0: 1639.0, 1: 1643.8. Samples: 9533212. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:44:21,443][43579] Avg episode reward: [(0, '258.500'), (1, '274.540')] [2023-10-12 20:44:22,194][44959] Updated weights for policy 1, policy_version 18660 (0.0009) [2023-10-12 20:44:22,557][44959] Updated weights for policy 1, policy_version 18670 (0.0007) [2023-10-12 20:44:22,923][44959] Updated weights for policy 1, policy_version 18680 (0.0009) [2023-10-12 20:44:23,134][44958] Updated weights for policy 0, policy_version 18570 (0.0007) [2023-10-12 20:44:23,504][44958] Updated weights for policy 0, policy_version 18580 (0.0009) [2023-10-12 20:44:23,875][44958] Updated weights for policy 0, policy_version 18590 (0.0009) [2023-10-12 20:44:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38174720. Throughput: 0: 1635.5, 1: 1649.6. Samples: 9553610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:44:26,444][43579] Avg episode reward: [(0, '255.230'), (1, '270.450')] [2023-10-12 20:44:26,819][44959] Updated weights for policy 1, policy_version 18690 (0.0009) [2023-10-12 20:44:27,195][44959] Updated weights for policy 1, policy_version 18700 (0.0007) [2023-10-12 20:44:27,571][44959] Updated weights for policy 1, policy_version 18710 (0.0007) [2023-10-12 20:44:27,940][44959] Updated weights for policy 1, policy_version 18720 (0.0007) [2023-10-12 20:44:27,985][44958] Updated weights for policy 0, policy_version 18600 (0.0008) [2023-10-12 20:44:28,347][44958] Updated weights for policy 0, policy_version 18610 (0.0007) [2023-10-12 20:44:28,725][44958] Updated weights for policy 0, policy_version 18620 (0.0007) [2023-10-12 20:44:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38240256. Throughput: 0: 1641.3, 1: 1646.9. Samples: 9574032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:44:31,444][43579] Avg episode reward: [(0, '259.710'), (1, '272.230')] [2023-10-12 20:44:32,230][44959] Updated weights for policy 1, policy_version 18730 (0.0007) [2023-10-12 20:44:32,607][44959] Updated weights for policy 1, policy_version 18740 (0.0007) [2023-10-12 20:44:32,975][44959] Updated weights for policy 1, policy_version 18750 (0.0008) [2023-10-12 20:44:33,069][44958] Updated weights for policy 0, policy_version 18630 (0.0007) [2023-10-12 20:44:33,434][44958] Updated weights for policy 0, policy_version 18640 (0.0010) [2023-10-12 20:44:33,819][44958] Updated weights for policy 0, policy_version 18650 (0.0011) [2023-10-12 20:44:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38305792. Throughput: 0: 1639.4, 1: 1648.0. Samples: 9582750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:44:36,443][43579] Avg episode reward: [(0, '263.880'), (1, '269.620')] [2023-10-12 20:44:37,207][44959] Updated weights for policy 1, policy_version 18760 (0.0008) [2023-10-12 20:44:37,582][44959] Updated weights for policy 1, policy_version 18770 (0.0007) [2023-10-12 20:44:37,951][44959] Updated weights for policy 1, policy_version 18780 (0.0007) [2023-10-12 20:44:38,044][44958] Updated weights for policy 0, policy_version 18660 (0.0007) [2023-10-12 20:44:38,410][44958] Updated weights for policy 0, policy_version 18670 (0.0008) [2023-10-12 20:44:38,791][44958] Updated weights for policy 0, policy_version 18680 (0.0008) [2023-10-12 20:44:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38371328. Throughput: 0: 1637.8, 1: 1650.9. Samples: 9603196. Policy #0 lag: (min: 11.0, avg: 12.3, max: 35.0) [2023-10-12 20:44:41,444][43579] Avg episode reward: [(0, '259.590'), (1, '266.310')] [2023-10-12 20:44:42,062][44959] Updated weights for policy 1, policy_version 18790 (0.0008) [2023-10-12 20:44:42,431][44959] Updated weights for policy 1, policy_version 18800 (0.0008) [2023-10-12 20:44:42,707][44958] Updated weights for policy 0, policy_version 18690 (0.0009) [2023-10-12 20:44:42,794][44959] Updated weights for policy 1, policy_version 18810 (0.0008) [2023-10-12 20:44:43,072][44958] Updated weights for policy 0, policy_version 18700 (0.0007) [2023-10-12 20:44:43,438][44958] Updated weights for policy 0, policy_version 18710 (0.0009) [2023-10-12 20:44:43,810][44958] Updated weights for policy 0, policy_version 18720 (0.0008) [2023-10-12 20:44:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38436864. Throughput: 0: 1646.1, 1: 1653.1. Samples: 9623598. Policy #0 lag: (min: 11.0, avg: 12.3, max: 35.0) [2023-10-12 20:44:46,443][43579] Avg episode reward: [(0, '263.380'), (1, '265.650')] [2023-10-12 20:44:46,906][44959] Updated weights for policy 1, policy_version 18820 (0.0008) [2023-10-12 20:44:47,271][44959] Updated weights for policy 1, policy_version 18830 (0.0009) [2023-10-12 20:44:47,632][44959] Updated weights for policy 1, policy_version 18840 (0.0008) [2023-10-12 20:44:48,112][44958] Updated weights for policy 0, policy_version 18730 (0.0009) [2023-10-12 20:44:48,475][44958] Updated weights for policy 0, policy_version 18740 (0.0011) [2023-10-12 20:44:48,857][44958] Updated weights for policy 0, policy_version 18750 (0.0009) [2023-10-12 20:44:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 38502400. Throughput: 0: 1646.1, 1: 1651.8. Samples: 9632336. Policy #0 lag: (min: 11.0, avg: 12.3, max: 35.0) [2023-10-12 20:44:51,443][43579] Avg episode reward: [(0, '269.760'), (1, '266.770')] [2023-10-12 20:44:51,815][44959] Updated weights for policy 1, policy_version 18850 (0.0008) [2023-10-12 20:44:52,185][44959] Updated weights for policy 1, policy_version 18860 (0.0011) [2023-10-12 20:44:52,558][44959] Updated weights for policy 1, policy_version 18870 (0.0010) [2023-10-12 20:44:52,918][44959] Updated weights for policy 1, policy_version 18880 (0.0010) [2023-10-12 20:44:52,963][44958] Updated weights for policy 0, policy_version 18760 (0.0008) [2023-10-12 20:44:53,343][44958] Updated weights for policy 0, policy_version 18770 (0.0009) [2023-10-12 20:44:53,706][44958] Updated weights for policy 0, policy_version 18780 (0.0009) [2023-10-12 20:44:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 38567936. Throughput: 0: 1643.0, 1: 1669.8. Samples: 9652942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:44:56,443][43579] Avg episode reward: [(0, '268.640'), (1, '266.760')] [2023-10-12 20:44:56,887][44959] Updated weights for policy 1, policy_version 18890 (0.0008) [2023-10-12 20:44:57,256][44959] Updated weights for policy 1, policy_version 18900 (0.0007) [2023-10-12 20:44:57,626][44959] Updated weights for policy 1, policy_version 18910 (0.0010) [2023-10-12 20:44:58,151][44958] Updated weights for policy 0, policy_version 18790 (0.0008) [2023-10-12 20:44:58,541][44958] Updated weights for policy 0, policy_version 18800 (0.0008) [2023-10-12 20:44:58,914][44958] Updated weights for policy 0, policy_version 18810 (0.0008) [2023-10-12 20:45:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38633472. Throughput: 0: 1640.0, 1: 1667.1. Samples: 9673182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:45:01,443][43579] Avg episode reward: [(0, '268.350'), (1, '272.420')] [2023-10-12 20:45:01,768][44959] Updated weights for policy 1, policy_version 18920 (0.0008) [2023-10-12 20:45:02,138][44959] Updated weights for policy 1, policy_version 18930 (0.0011) [2023-10-12 20:45:02,502][44959] Updated weights for policy 1, policy_version 18940 (0.0010) [2023-10-12 20:45:03,102][44958] Updated weights for policy 0, policy_version 18820 (0.0009) [2023-10-12 20:45:03,466][44958] Updated weights for policy 0, policy_version 18830 (0.0009) [2023-10-12 20:45:03,844][44958] Updated weights for policy 0, policy_version 18840 (0.0009) [2023-10-12 20:45:06,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38699008. Throughput: 0: 1638.5, 1: 1665.9. Samples: 9681912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:45:06,444][43579] Avg episode reward: [(0, '269.220'), (1, '274.610')] [2023-10-12 20:45:06,624][44959] Updated weights for policy 1, policy_version 18950 (0.0010) [2023-10-12 20:45:06,990][44959] Updated weights for policy 1, policy_version 18960 (0.0011) [2023-10-12 20:45:07,355][44959] Updated weights for policy 1, policy_version 18970 (0.0010) [2023-10-12 20:45:08,039][44958] Updated weights for policy 0, policy_version 18850 (0.0009) [2023-10-12 20:45:08,411][44958] Updated weights for policy 0, policy_version 18860 (0.0010) [2023-10-12 20:45:08,798][44958] Updated weights for policy 0, policy_version 18870 (0.0008) [2023-10-12 20:45:09,162][44958] Updated weights for policy 0, policy_version 18880 (0.0010) [2023-10-12 20:45:11,350][44959] Updated weights for policy 1, policy_version 18980 (0.0008) [2023-10-12 20:45:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 38764544. Throughput: 0: 1634.5, 1: 1668.4. Samples: 9702242. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-10-12 20:45:11,444][43579] Avg episode reward: [(0, '269.440'), (1, '271.320')] [2023-10-12 20:45:11,721][44959] Updated weights for policy 1, policy_version 18990 (0.0008) [2023-10-12 20:45:12,086][44959] Updated weights for policy 1, policy_version 19000 (0.0008) [2023-10-12 20:45:13,196][44958] Updated weights for policy 0, policy_version 18890 (0.0009) [2023-10-12 20:45:13,567][44958] Updated weights for policy 0, policy_version 18900 (0.0009) [2023-10-12 20:45:13,936][44958] Updated weights for policy 0, policy_version 18910 (0.0009) [2023-10-12 20:45:16,102][44959] Updated weights for policy 1, policy_version 19010 (0.0008) [2023-10-12 20:45:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38830080. Throughput: 0: 1634.8, 1: 1673.6. Samples: 9722908. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-10-12 20:45:16,444][43579] Avg episode reward: [(0, '265.430'), (1, '270.350')] [2023-10-12 20:45:16,473][44959] Updated weights for policy 1, policy_version 19020 (0.0009) [2023-10-12 20:45:16,844][44959] Updated weights for policy 1, policy_version 19030 (0.0009) [2023-10-12 20:45:17,216][44959] Updated weights for policy 1, policy_version 19040 (0.0008) [2023-10-12 20:45:18,163][44958] Updated weights for policy 0, policy_version 18920 (0.0009) [2023-10-12 20:45:18,538][44958] Updated weights for policy 0, policy_version 18930 (0.0008) [2023-10-12 20:45:18,917][44958] Updated weights for policy 0, policy_version 18940 (0.0009) [2023-10-12 20:45:21,191][44959] Updated weights for policy 1, policy_version 19050 (0.0010) [2023-10-12 20:45:21,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38895616. Throughput: 0: 1634.5, 1: 1677.2. Samples: 9731774. Policy #0 lag: (min: 23.0, avg: 31.0, max: 55.0) [2023-10-12 20:45:21,443][43579] Avg episode reward: [(0, '258.900'), (1, '266.630')] [2023-10-12 20:45:21,558][44959] Updated weights for policy 1, policy_version 19060 (0.0009) [2023-10-12 20:45:21,932][44959] Updated weights for policy 1, policy_version 19070 (0.0007) [2023-10-12 20:45:23,183][44958] Updated weights for policy 0, policy_version 18950 (0.0008) [2023-10-12 20:45:23,551][44958] Updated weights for policy 0, policy_version 18960 (0.0010) [2023-10-12 20:45:23,923][44958] Updated weights for policy 0, policy_version 18970 (0.0009) [2023-10-12 20:45:26,157][44959] Updated weights for policy 1, policy_version 19080 (0.0009) [2023-10-12 20:45:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 38961152. Throughput: 0: 1632.6, 1: 1675.5. Samples: 9752060. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 20:45:26,444][43579] Avg episode reward: [(0, '261.500'), (1, '262.950')] [2023-10-12 20:45:26,531][44959] Updated weights for policy 1, policy_version 19090 (0.0008) [2023-10-12 20:45:26,908][44959] Updated weights for policy 1, policy_version 19100 (0.0008) [2023-10-12 20:45:27,774][44958] Updated weights for policy 0, policy_version 18980 (0.0009) [2023-10-12 20:45:28,139][44958] Updated weights for policy 0, policy_version 18990 (0.0009) [2023-10-12 20:45:28,510][44958] Updated weights for policy 0, policy_version 19000 (0.0010) [2023-10-12 20:45:31,047][44959] Updated weights for policy 1, policy_version 19110 (0.0007) [2023-10-12 20:45:31,409][44959] Updated weights for policy 1, policy_version 19120 (0.0008) [2023-10-12 20:45:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39026688. Throughput: 0: 1630.5, 1: 1670.2. Samples: 9772132. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 20:45:31,443][43579] Avg episode reward: [(0, '260.890'), (1, '258.770')] [2023-10-12 20:45:31,784][44959] Updated weights for policy 1, policy_version 19130 (0.0009) [2023-10-12 20:45:32,915][44958] Updated weights for policy 0, policy_version 19010 (0.0008) [2023-10-12 20:45:33,289][44958] Updated weights for policy 0, policy_version 19020 (0.0007) [2023-10-12 20:45:33,657][44958] Updated weights for policy 0, policy_version 19030 (0.0008) [2023-10-12 20:45:34,030][44958] Updated weights for policy 0, policy_version 19040 (0.0007) [2023-10-12 20:45:35,888][44959] Updated weights for policy 1, policy_version 19140 (0.0009) [2023-10-12 20:45:36,249][44959] Updated weights for policy 1, policy_version 19150 (0.0008) [2023-10-12 20:45:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39092224. Throughput: 0: 1629.1, 1: 1678.4. Samples: 9781174. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 20:45:36,443][43579] Avg episode reward: [(0, '264.760'), (1, '260.960')] [2023-10-12 20:45:36,625][44959] Updated weights for policy 1, policy_version 19160 (0.0007) [2023-10-12 20:45:38,141][44958] Updated weights for policy 0, policy_version 19050 (0.0007) [2023-10-12 20:45:38,519][44958] Updated weights for policy 0, policy_version 19060 (0.0008) [2023-10-12 20:45:38,891][44958] Updated weights for policy 0, policy_version 19070 (0.0010) [2023-10-12 20:45:40,894][44959] Updated weights for policy 1, policy_version 19170 (0.0008) [2023-10-12 20:45:41,310][44959] Updated weights for policy 1, policy_version 19180 (0.0007) [2023-10-12 20:45:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39157760. Throughput: 0: 1637.4, 1: 1668.8. Samples: 9801720. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 20:45:41,443][43579] Avg episode reward: [(0, '265.090'), (1, '256.850')] [2023-10-12 20:45:41,678][44959] Updated weights for policy 1, policy_version 19190 (0.0007) [2023-10-12 20:45:42,052][44959] Updated weights for policy 1, policy_version 19200 (0.0008) [2023-10-12 20:45:42,994][44958] Updated weights for policy 0, policy_version 19080 (0.0008) [2023-10-12 20:45:43,365][44958] Updated weights for policy 0, policy_version 19090 (0.0007) [2023-10-12 20:45:43,746][44958] Updated weights for policy 0, policy_version 19100 (0.0008) [2023-10-12 20:45:46,168][44959] Updated weights for policy 1, policy_version 19210 (0.0007) [2023-10-12 20:45:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39223296. Throughput: 0: 1644.5, 1: 1659.2. Samples: 9821848. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 20:45:46,443][43579] Avg episode reward: [(0, '270.670'), (1, '261.190')] [2023-10-12 20:45:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000019104_19562496.pth... [2023-10-12 20:45:46,488][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000017568_17989632.pth [2023-10-12 20:45:46,537][44959] Updated weights for policy 1, policy_version 19220 (0.0008) [2023-10-12 20:45:46,911][44959] Updated weights for policy 1, policy_version 19230 (0.0009) [2023-10-12 20:45:46,983][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000019232_19693568.pth... [2023-10-12 20:45:47,015][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000017664_18087936.pth [2023-10-12 20:45:47,762][44958] Updated weights for policy 0, policy_version 19110 (0.0008) [2023-10-12 20:45:48,127][44958] Updated weights for policy 0, policy_version 19120 (0.0008) [2023-10-12 20:45:48,503][44958] Updated weights for policy 0, policy_version 19130 (0.0008) [2023-10-12 20:45:51,055][44959] Updated weights for policy 1, policy_version 19240 (0.0008) [2023-10-12 20:45:51,423][44959] Updated weights for policy 1, policy_version 19250 (0.0010) [2023-10-12 20:45:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39288832. Throughput: 0: 1645.5, 1: 1671.9. Samples: 9831194. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 20:45:51,443][43579] Avg episode reward: [(0, '275.630'), (1, '266.310')] [2023-10-12 20:45:51,784][44959] Updated weights for policy 1, policy_version 19260 (0.0008) [2023-10-12 20:45:52,603][44958] Updated weights for policy 0, policy_version 19140 (0.0008) [2023-10-12 20:45:52,985][44958] Updated weights for policy 0, policy_version 19150 (0.0007) [2023-10-12 20:45:53,353][44958] Updated weights for policy 0, policy_version 19160 (0.0009) [2023-10-12 20:45:55,813][44959] Updated weights for policy 1, policy_version 19270 (0.0007) [2023-10-12 20:45:56,186][44959] Updated weights for policy 1, policy_version 19280 (0.0008) [2023-10-12 20:45:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39354368. Throughput: 0: 1654.8, 1: 1663.4. Samples: 9851560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:45:56,444][43579] Avg episode reward: [(0, '272.090'), (1, '267.160')] [2023-10-12 20:45:56,569][44959] Updated weights for policy 1, policy_version 19290 (0.0008) [2023-10-12 20:45:57,526][44958] Updated weights for policy 0, policy_version 19170 (0.0009) [2023-10-12 20:45:57,895][44958] Updated weights for policy 0, policy_version 19180 (0.0009) [2023-10-12 20:45:58,273][44958] Updated weights for policy 0, policy_version 19190 (0.0009) [2023-10-12 20:45:58,645][44958] Updated weights for policy 0, policy_version 19200 (0.0007) [2023-10-12 20:46:00,778][44959] Updated weights for policy 1, policy_version 19300 (0.0009) [2023-10-12 20:46:01,149][44959] Updated weights for policy 1, policy_version 19310 (0.0009) [2023-10-12 20:46:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39419904. Throughput: 0: 1651.8, 1: 1646.0. Samples: 9871306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:46:01,443][43579] Avg episode reward: [(0, '273.330'), (1, '267.140')] [2023-10-12 20:46:01,518][44959] Updated weights for policy 1, policy_version 19320 (0.0009) [2023-10-12 20:46:02,846][44958] Updated weights for policy 0, policy_version 19210 (0.0008) [2023-10-12 20:46:03,224][44958] Updated weights for policy 0, policy_version 19220 (0.0009) [2023-10-12 20:46:03,599][44958] Updated weights for policy 0, policy_version 19230 (0.0010) [2023-10-12 20:46:05,527][44959] Updated weights for policy 1, policy_version 19330 (0.0008) [2023-10-12 20:46:05,899][44959] Updated weights for policy 1, policy_version 19340 (0.0007) [2023-10-12 20:46:06,264][44959] Updated weights for policy 1, policy_version 19350 (0.0007) [2023-10-12 20:46:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39485440. Throughput: 0: 1654.6, 1: 1652.9. Samples: 9880612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:46:06,443][43579] Avg episode reward: [(0, '274.340'), (1, '270.880')] [2023-10-12 20:46:06,630][44959] Updated weights for policy 1, policy_version 19360 (0.0009) [2023-10-12 20:46:07,617][44958] Updated weights for policy 0, policy_version 19240 (0.0008) [2023-10-12 20:46:07,997][44958] Updated weights for policy 0, policy_version 19250 (0.0007) [2023-10-12 20:46:08,359][44958] Updated weights for policy 0, policy_version 19260 (0.0007) [2023-10-12 20:46:10,862][44959] Updated weights for policy 1, policy_version 19370 (0.0008) [2023-10-12 20:46:11,220][44959] Updated weights for policy 1, policy_version 19380 (0.0009) [2023-10-12 20:46:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 39550976. Throughput: 0: 1658.0, 1: 1646.1. Samples: 9900742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:46:11,443][43579] Avg episode reward: [(0, '274.330'), (1, '272.950')] [2023-10-12 20:46:11,588][44959] Updated weights for policy 1, policy_version 19390 (0.0008) [2023-10-12 20:46:12,488][44958] Updated weights for policy 0, policy_version 19270 (0.0007) [2023-10-12 20:46:12,860][44958] Updated weights for policy 0, policy_version 19280 (0.0008) [2023-10-12 20:46:13,229][44958] Updated weights for policy 0, policy_version 19290 (0.0007) [2023-10-12 20:46:15,711][44959] Updated weights for policy 1, policy_version 19400 (0.0008) [2023-10-12 20:46:16,075][44959] Updated weights for policy 1, policy_version 19410 (0.0010) [2023-10-12 20:46:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39616512. Throughput: 0: 1658.4, 1: 1639.6. Samples: 9920546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:46:16,444][43579] Avg episode reward: [(0, '272.190'), (1, '275.540')] [2023-10-12 20:46:16,445][44959] Updated weights for policy 1, policy_version 19420 (0.0010) [2023-10-12 20:46:17,443][44958] Updated weights for policy 0, policy_version 19300 (0.0010) [2023-10-12 20:46:17,820][44958] Updated weights for policy 0, policy_version 19310 (0.0010) [2023-10-12 20:46:18,200][44958] Updated weights for policy 0, policy_version 19320 (0.0008) [2023-10-12 20:46:20,556][44959] Updated weights for policy 1, policy_version 19430 (0.0008) [2023-10-12 20:46:20,927][44959] Updated weights for policy 1, policy_version 19440 (0.0008) [2023-10-12 20:46:21,293][44959] Updated weights for policy 1, policy_version 19450 (0.0009) [2023-10-12 20:46:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39682048. Throughput: 0: 1659.6, 1: 1648.4. Samples: 9930034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:46:21,443][43579] Avg episode reward: [(0, '270.530'), (1, '270.440')] [2023-10-12 20:46:22,318][44958] Updated weights for policy 0, policy_version 19330 (0.0008) [2023-10-12 20:46:22,699][44958] Updated weights for policy 0, policy_version 19340 (0.0007) [2023-10-12 20:46:23,074][44958] Updated weights for policy 0, policy_version 19350 (0.0007) [2023-10-12 20:46:23,444][44958] Updated weights for policy 0, policy_version 19360 (0.0008) [2023-10-12 20:46:25,554][44959] Updated weights for policy 1, policy_version 19460 (0.0009) [2023-10-12 20:46:25,942][44959] Updated weights for policy 1, policy_version 19470 (0.0008) [2023-10-12 20:46:26,315][44959] Updated weights for policy 1, policy_version 19480 (0.0007) [2023-10-12 20:46:26,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39747584. Throughput: 0: 1652.8, 1: 1650.5. Samples: 9950370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:46:26,443][43579] Avg episode reward: [(0, '273.200'), (1, '268.610')] [2023-10-12 20:46:27,771][44958] Updated weights for policy 0, policy_version 19370 (0.0009) [2023-10-12 20:46:28,152][44958] Updated weights for policy 0, policy_version 19380 (0.0008) [2023-10-12 20:46:28,514][44958] Updated weights for policy 0, policy_version 19390 (0.0010) [2023-10-12 20:46:30,499][44959] Updated weights for policy 1, policy_version 19490 (0.0008) [2023-10-12 20:46:30,869][44959] Updated weights for policy 1, policy_version 19500 (0.0008) [2023-10-12 20:46:31,229][44959] Updated weights for policy 1, policy_version 19510 (0.0009) [2023-10-12 20:46:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39813120. Throughput: 0: 1644.7, 1: 1643.2. Samples: 9969802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:46:31,443][43579] Avg episode reward: [(0, '266.390'), (1, '270.650')] [2023-10-12 20:46:31,596][44959] Updated weights for policy 1, policy_version 19520 (0.0010) [2023-10-12 20:46:32,595][44958] Updated weights for policy 0, policy_version 19400 (0.0009) [2023-10-12 20:46:32,970][44958] Updated weights for policy 0, policy_version 19410 (0.0009) [2023-10-12 20:46:33,346][44958] Updated weights for policy 0, policy_version 19420 (0.0008) [2023-10-12 20:46:35,694][44959] Updated weights for policy 1, policy_version 19530 (0.0007) [2023-10-12 20:46:36,056][44959] Updated weights for policy 1, policy_version 19540 (0.0010) [2023-10-12 20:46:36,430][44959] Updated weights for policy 1, policy_version 19550 (0.0010) [2023-10-12 20:46:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39878656. Throughput: 0: 1646.2, 1: 1647.4. Samples: 9979406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:46:36,444][43579] Avg episode reward: [(0, '264.780'), (1, '266.340')] [2023-10-12 20:46:37,446][44958] Updated weights for policy 0, policy_version 19430 (0.0009) [2023-10-12 20:46:37,829][44958] Updated weights for policy 0, policy_version 19440 (0.0007) [2023-10-12 20:46:38,200][44958] Updated weights for policy 0, policy_version 19450 (0.0010) [2023-10-12 20:46:40,654][44959] Updated weights for policy 1, policy_version 19560 (0.0009) [2023-10-12 20:46:41,025][44959] Updated weights for policy 1, policy_version 19570 (0.0008) [2023-10-12 20:46:41,401][44959] Updated weights for policy 1, policy_version 19580 (0.0009) [2023-10-12 20:46:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 39944192. Throughput: 0: 1643.0, 1: 1652.2. Samples: 9999846. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:46:41,444][43579] Avg episode reward: [(0, '264.590'), (1, '265.790')] [2023-10-12 20:46:42,538][44958] Updated weights for policy 0, policy_version 19460 (0.0007) [2023-10-12 20:46:42,912][44958] Updated weights for policy 0, policy_version 19470 (0.0007) [2023-10-12 20:46:43,287][44958] Updated weights for policy 0, policy_version 19480 (0.0009) [2023-10-12 20:46:45,576][44959] Updated weights for policy 1, policy_version 19590 (0.0009) [2023-10-12 20:46:45,944][44959] Updated weights for policy 1, policy_version 19600 (0.0009) [2023-10-12 20:46:46,317][44959] Updated weights for policy 1, policy_version 19610 (0.0010) [2023-10-12 20:46:46,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 40009728. Throughput: 0: 1648.5, 1: 1643.0. Samples: 10019422. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:46:46,443][43579] Avg episode reward: [(0, '264.830'), (1, '264.360')] [2023-10-12 20:46:47,322][44958] Updated weights for policy 0, policy_version 19490 (0.0010) [2023-10-12 20:46:47,697][44958] Updated weights for policy 0, policy_version 19500 (0.0008) [2023-10-12 20:46:48,071][44958] Updated weights for policy 0, policy_version 19510 (0.0010) [2023-10-12 20:46:48,445][44958] Updated weights for policy 0, policy_version 19520 (0.0009) [2023-10-12 20:46:50,505][44959] Updated weights for policy 1, policy_version 19620 (0.0009) [2023-10-12 20:46:50,876][44959] Updated weights for policy 1, policy_version 19630 (0.0008) [2023-10-12 20:46:51,241][44959] Updated weights for policy 1, policy_version 19640 (0.0007) [2023-10-12 20:46:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 40075264. Throughput: 0: 1647.2, 1: 1648.4. Samples: 10028912. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-12 20:46:51,444][43579] Avg episode reward: [(0, '267.940'), (1, '270.510')] [2023-10-12 20:46:52,587][44958] Updated weights for policy 0, policy_version 19530 (0.0009) [2023-10-12 20:46:52,967][44958] Updated weights for policy 0, policy_version 19540 (0.0010) [2023-10-12 20:46:53,345][44958] Updated weights for policy 0, policy_version 19550 (0.0008) [2023-10-12 20:46:55,566][44959] Updated weights for policy 1, policy_version 19650 (0.0007) [2023-10-12 20:46:55,937][44959] Updated weights for policy 1, policy_version 19660 (0.0008) [2023-10-12 20:46:56,314][44959] Updated weights for policy 1, policy_version 19670 (0.0009) [2023-10-12 20:46:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 40140800. Throughput: 0: 1651.3, 1: 1655.2. Samples: 10049536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:46:56,443][43579] Avg episode reward: [(0, '270.830'), (1, '274.550')] [2023-10-12 20:46:56,672][44959] Updated weights for policy 1, policy_version 19680 (0.0009) [2023-10-12 20:46:57,545][44958] Updated weights for policy 0, policy_version 19560 (0.0010) [2023-10-12 20:46:57,917][44958] Updated weights for policy 0, policy_version 19570 (0.0008) [2023-10-12 20:46:58,285][44958] Updated weights for policy 0, policy_version 19580 (0.0009) [2023-10-12 20:47:00,754][44959] Updated weights for policy 1, policy_version 19690 (0.0008) [2023-10-12 20:47:01,128][44959] Updated weights for policy 1, policy_version 19700 (0.0009) [2023-10-12 20:47:01,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 40206336. Throughput: 0: 1648.0, 1: 1654.1. Samples: 10069140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:47:01,443][43579] Avg episode reward: [(0, '274.360'), (1, '278.670')] [2023-10-12 20:47:01,494][44959] Updated weights for policy 1, policy_version 19710 (0.0010) [2023-10-12 20:47:02,473][44958] Updated weights for policy 0, policy_version 19590 (0.0010) [2023-10-12 20:47:02,844][44958] Updated weights for policy 0, policy_version 19600 (0.0009) [2023-10-12 20:47:03,227][44958] Updated weights for policy 0, policy_version 19610 (0.0007) [2023-10-12 20:47:05,603][44959] Updated weights for policy 1, policy_version 19720 (0.0008) [2023-10-12 20:47:05,982][44959] Updated weights for policy 1, policy_version 19730 (0.0009) [2023-10-12 20:47:06,344][44959] Updated weights for policy 1, policy_version 19740 (0.0008) [2023-10-12 20:47:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 40271872. Throughput: 0: 1645.2, 1: 1653.4. Samples: 10078472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:47:06,443][43579] Avg episode reward: [(0, '272.480'), (1, '279.080')] [2023-10-12 20:47:07,404][44958] Updated weights for policy 0, policy_version 19620 (0.0010) [2023-10-12 20:47:07,771][44958] Updated weights for policy 0, policy_version 19630 (0.0008) [2023-10-12 20:47:08,155][44958] Updated weights for policy 0, policy_version 19640 (0.0008) [2023-10-12 20:47:10,433][44959] Updated weights for policy 1, policy_version 19750 (0.0008) [2023-10-12 20:47:10,795][44959] Updated weights for policy 1, policy_version 19760 (0.0010) [2023-10-12 20:47:11,174][44959] Updated weights for policy 1, policy_version 19770 (0.0011) [2023-10-12 20:47:11,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 40370176. Throughput: 0: 1641.4, 1: 1655.9. Samples: 10098746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:47:11,444][43579] Avg episode reward: [(0, '272.000'), (1, '278.420')] [2023-10-12 20:47:12,388][44958] Updated weights for policy 0, policy_version 19650 (0.0008) [2023-10-12 20:47:12,792][44958] Updated weights for policy 0, policy_version 19660 (0.0009) [2023-10-12 20:47:13,165][44958] Updated weights for policy 0, policy_version 19670 (0.0009) [2023-10-12 20:47:13,539][44958] Updated weights for policy 0, policy_version 19680 (0.0009) [2023-10-12 20:47:15,211][44959] Updated weights for policy 1, policy_version 19780 (0.0008) [2023-10-12 20:47:15,589][44959] Updated weights for policy 1, policy_version 19790 (0.0008) [2023-10-12 20:47:15,948][44959] Updated weights for policy 1, policy_version 19800 (0.0008) [2023-10-12 20:47:16,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 40435712. Throughput: 0: 1646.6, 1: 1648.7. Samples: 10118090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:47:16,443][43579] Avg episode reward: [(0, '271.820'), (1, '275.980')] [2023-10-12 20:47:17,642][44958] Updated weights for policy 0, policy_version 19690 (0.0009) [2023-10-12 20:47:18,020][44958] Updated weights for policy 0, policy_version 19700 (0.0010) [2023-10-12 20:47:18,394][44958] Updated weights for policy 0, policy_version 19710 (0.0008) [2023-10-12 20:47:19,958][44959] Updated weights for policy 1, policy_version 19810 (0.0009) [2023-10-12 20:47:20,332][44959] Updated weights for policy 1, policy_version 19820 (0.0008) [2023-10-12 20:47:20,699][44959] Updated weights for policy 1, policy_version 19830 (0.0008) [2023-10-12 20:47:21,079][44959] Updated weights for policy 1, policy_version 19840 (0.0008) [2023-10-12 20:47:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 40501248. Throughput: 0: 1644.6, 1: 1658.0. Samples: 10128022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:47:21,443][43579] Avg episode reward: [(0, '265.810'), (1, '271.570')] [2023-10-12 20:47:22,416][44958] Updated weights for policy 0, policy_version 19720 (0.0009) [2023-10-12 20:47:22,792][44958] Updated weights for policy 0, policy_version 19730 (0.0008) [2023-10-12 20:47:23,175][44958] Updated weights for policy 0, policy_version 19740 (0.0008) [2023-10-12 20:47:25,202][44959] Updated weights for policy 1, policy_version 19850 (0.0009) [2023-10-12 20:47:25,556][44959] Updated weights for policy 1, policy_version 19860 (0.0008) [2023-10-12 20:47:25,927][44959] Updated weights for policy 1, policy_version 19870 (0.0010) [2023-10-12 20:47:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 40566784. Throughput: 0: 1645.8, 1: 1646.6. Samples: 10148004. Policy #0 lag: (min: 31.0, avg: 48.8, max: 63.0) [2023-10-12 20:47:26,443][43579] Avg episode reward: [(0, '264.270'), (1, '268.890')] [2023-10-12 20:47:27,319][44958] Updated weights for policy 0, policy_version 19750 (0.0008) [2023-10-12 20:47:27,690][44958] Updated weights for policy 0, policy_version 19760 (0.0007) [2023-10-12 20:47:28,067][44958] Updated weights for policy 0, policy_version 19770 (0.0008) [2023-10-12 20:47:30,263][44959] Updated weights for policy 1, policy_version 19880 (0.0008) [2023-10-12 20:47:30,629][44959] Updated weights for policy 1, policy_version 19890 (0.0007) [2023-10-12 20:47:30,998][44959] Updated weights for policy 1, policy_version 19900 (0.0008) [2023-10-12 20:47:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 40632320. Throughput: 0: 1641.0, 1: 1650.3. Samples: 10167530. Policy #0 lag: (min: 31.0, avg: 48.8, max: 63.0) [2023-10-12 20:47:31,443][43579] Avg episode reward: [(0, '265.010'), (1, '267.990')] [2023-10-12 20:47:32,372][44958] Updated weights for policy 0, policy_version 19780 (0.0010) [2023-10-12 20:47:32,744][44958] Updated weights for policy 0, policy_version 19790 (0.0009) [2023-10-12 20:47:33,103][44958] Updated weights for policy 0, policy_version 19800 (0.0010) [2023-10-12 20:47:35,084][44959] Updated weights for policy 1, policy_version 19910 (0.0007) [2023-10-12 20:47:35,450][44959] Updated weights for policy 1, policy_version 19920 (0.0008) [2023-10-12 20:47:35,817][44959] Updated weights for policy 1, policy_version 19930 (0.0009) [2023-10-12 20:47:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 40697856. Throughput: 0: 1640.1, 1: 1657.1. Samples: 10177288. Policy #0 lag: (min: 31.0, avg: 48.8, max: 63.0) [2023-10-12 20:47:36,443][43579] Avg episode reward: [(0, '263.210'), (1, '272.430')] [2023-10-12 20:47:37,225][44958] Updated weights for policy 0, policy_version 19810 (0.0009) [2023-10-12 20:47:37,594][44958] Updated weights for policy 0, policy_version 19820 (0.0007) [2023-10-12 20:47:37,971][44958] Updated weights for policy 0, policy_version 19830 (0.0008) [2023-10-12 20:47:38,347][44958] Updated weights for policy 0, policy_version 19840 (0.0008) [2023-10-12 20:47:39,940][44959] Updated weights for policy 1, policy_version 19940 (0.0009) [2023-10-12 20:47:40,300][44959] Updated weights for policy 1, policy_version 19950 (0.0010) [2023-10-12 20:47:40,663][44959] Updated weights for policy 1, policy_version 19960 (0.0011) [2023-10-12 20:47:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 40763392. Throughput: 0: 1635.9, 1: 1649.5. Samples: 10197384. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-12 20:47:41,444][43579] Avg episode reward: [(0, '263.660'), (1, '274.070')] [2023-10-12 20:47:42,647][44958] Updated weights for policy 0, policy_version 19850 (0.0011) [2023-10-12 20:47:43,028][44958] Updated weights for policy 0, policy_version 19860 (0.0011) [2023-10-12 20:47:43,407][44958] Updated weights for policy 0, policy_version 19870 (0.0009) [2023-10-12 20:47:44,708][44959] Updated weights for policy 1, policy_version 19970 (0.0010) [2023-10-12 20:47:45,083][44959] Updated weights for policy 1, policy_version 19980 (0.0009) [2023-10-12 20:47:45,451][44959] Updated weights for policy 1, policy_version 19990 (0.0009) [2023-10-12 20:47:45,823][44959] Updated weights for policy 1, policy_version 20000 (0.0009) [2023-10-12 20:47:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13653.2, 300 sec: 13218.3). Total num frames: 40828928. Throughput: 0: 1635.9, 1: 1643.4. Samples: 10216710. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-12 20:47:46,444][43579] Avg episode reward: [(0, '259.100'), (1, '276.270')] [2023-10-12 20:47:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000020000_20480000.pth... [2023-10-12 20:47:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000019872_20348928.pth... [2023-10-12 20:47:46,497][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000018336_18776064.pth [2023-10-12 20:47:46,502][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000018432_18874368.pth [2023-10-12 20:47:47,489][44958] Updated weights for policy 0, policy_version 19880 (0.0008) [2023-10-12 20:47:47,858][44958] Updated weights for policy 0, policy_version 19890 (0.0008) [2023-10-12 20:47:48,238][44958] Updated weights for policy 0, policy_version 19900 (0.0009) [2023-10-12 20:47:50,072][44959] Updated weights for policy 1, policy_version 20010 (0.0007) [2023-10-12 20:47:50,446][44959] Updated weights for policy 1, policy_version 20020 (0.0007) [2023-10-12 20:47:50,814][44959] Updated weights for policy 1, policy_version 20030 (0.0008) [2023-10-12 20:47:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 40894464. Throughput: 0: 1638.8, 1: 1658.0. Samples: 10226826. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-12 20:47:51,444][43579] Avg episode reward: [(0, '259.310'), (1, '269.230')] [2023-10-12 20:47:52,363][44958] Updated weights for policy 0, policy_version 19910 (0.0008) [2023-10-12 20:47:52,748][44958] Updated weights for policy 0, policy_version 19920 (0.0009) [2023-10-12 20:47:53,132][44958] Updated weights for policy 0, policy_version 19930 (0.0008) [2023-10-12 20:47:55,201][44959] Updated weights for policy 1, policy_version 20040 (0.0008) [2023-10-12 20:47:55,565][44959] Updated weights for policy 1, policy_version 20050 (0.0010) [2023-10-12 20:47:55,943][44959] Updated weights for policy 1, policy_version 20060 (0.0008) [2023-10-12 20:47:56,443][43579] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 40960000. Throughput: 0: 1646.7, 1: 1648.9. Samples: 10247046. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-12 20:47:56,443][43579] Avg episode reward: [(0, '261.460'), (1, '267.730')] [2023-10-12 20:47:57,279][44958] Updated weights for policy 0, policy_version 19940 (0.0008) [2023-10-12 20:47:57,664][44958] Updated weights for policy 0, policy_version 19950 (0.0008) [2023-10-12 20:47:58,038][44958] Updated weights for policy 0, policy_version 19960 (0.0009) [2023-10-12 20:47:59,990][44959] Updated weights for policy 1, policy_version 20070 (0.0008) [2023-10-12 20:48:00,367][44959] Updated weights for policy 1, policy_version 20080 (0.0008) [2023-10-12 20:48:00,744][44959] Updated weights for policy 1, policy_version 20090 (0.0010) [2023-10-12 20:48:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 41025536. Throughput: 0: 1641.1, 1: 1649.4. Samples: 10266162. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-12 20:48:01,443][43579] Avg episode reward: [(0, '263.830'), (1, '269.640')] [2023-10-12 20:48:01,999][44958] Updated weights for policy 0, policy_version 19970 (0.0010) [2023-10-12 20:48:02,368][44958] Updated weights for policy 0, policy_version 19980 (0.0010) [2023-10-12 20:48:02,740][44958] Updated weights for policy 0, policy_version 19990 (0.0010) [2023-10-12 20:48:03,119][44958] Updated weights for policy 0, policy_version 20000 (0.0008) [2023-10-12 20:48:04,710][44959] Updated weights for policy 1, policy_version 20100 (0.0008) [2023-10-12 20:48:05,081][44959] Updated weights for policy 1, policy_version 20110 (0.0007) [2023-10-12 20:48:05,461][44959] Updated weights for policy 1, policy_version 20120 (0.0007) [2023-10-12 20:48:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 41091072. Throughput: 0: 1639.1, 1: 1658.7. Samples: 10276420. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-12 20:48:06,444][43579] Avg episode reward: [(0, '267.300'), (1, '266.450')] [2023-10-12 20:48:07,423][44958] Updated weights for policy 0, policy_version 20010 (0.0010) [2023-10-12 20:48:07,795][44958] Updated weights for policy 0, policy_version 20020 (0.0011) [2023-10-12 20:48:08,168][44958] Updated weights for policy 0, policy_version 20030 (0.0010) [2023-10-12 20:48:09,591][44959] Updated weights for policy 1, policy_version 20130 (0.0008) [2023-10-12 20:48:09,950][44959] Updated weights for policy 1, policy_version 20140 (0.0009) [2023-10-12 20:48:10,326][44959] Updated weights for policy 1, policy_version 20150 (0.0008) [2023-10-12 20:48:10,696][44959] Updated weights for policy 1, policy_version 20160 (0.0007) [2023-10-12 20:48:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41156608. Throughput: 0: 1634.5, 1: 1656.0. Samples: 10296080. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 20:48:11,444][43579] Avg episode reward: [(0, '271.580'), (1, '263.700')] [2023-10-12 20:48:12,557][44958] Updated weights for policy 0, policy_version 20040 (0.0010) [2023-10-12 20:48:12,935][44958] Updated weights for policy 0, policy_version 20050 (0.0009) [2023-10-12 20:48:13,311][44958] Updated weights for policy 0, policy_version 20060 (0.0011) [2023-10-12 20:48:14,781][44959] Updated weights for policy 1, policy_version 20170 (0.0008) [2023-10-12 20:48:15,155][44959] Updated weights for policy 1, policy_version 20180 (0.0010) [2023-10-12 20:48:15,515][44959] Updated weights for policy 1, policy_version 20190 (0.0010) [2023-10-12 20:48:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 41222144. Throughput: 0: 1636.3, 1: 1654.5. Samples: 10315616. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 20:48:16,444][43579] Avg episode reward: [(0, '271.150'), (1, '265.930')] [2023-10-12 20:48:17,540][44958] Updated weights for policy 0, policy_version 20070 (0.0010) [2023-10-12 20:48:17,920][44958] Updated weights for policy 0, policy_version 20080 (0.0008) [2023-10-12 20:48:18,294][44958] Updated weights for policy 0, policy_version 20090 (0.0009) [2023-10-12 20:48:19,693][44959] Updated weights for policy 1, policy_version 20200 (0.0010) [2023-10-12 20:48:20,059][44959] Updated weights for policy 1, policy_version 20210 (0.0009) [2023-10-12 20:48:20,426][44959] Updated weights for policy 1, policy_version 20220 (0.0009) [2023-10-12 20:48:21,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41287680. Throughput: 0: 1636.3, 1: 1655.5. Samples: 10325416. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 20:48:21,443][43579] Avg episode reward: [(0, '276.060'), (1, '270.380')] [2023-10-12 20:48:22,515][44958] Updated weights for policy 0, policy_version 20100 (0.0008) [2023-10-12 20:48:22,883][44958] Updated weights for policy 0, policy_version 20110 (0.0007) [2023-10-12 20:48:23,251][44958] Updated weights for policy 0, policy_version 20120 (0.0007) [2023-10-12 20:48:24,722][44959] Updated weights for policy 1, policy_version 20230 (0.0008) [2023-10-12 20:48:25,086][44959] Updated weights for policy 1, policy_version 20240 (0.0007) [2023-10-12 20:48:25,463][44959] Updated weights for policy 1, policy_version 20250 (0.0009) [2023-10-12 20:48:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41353216. Throughput: 0: 1637.9, 1: 1653.6. Samples: 10345502. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-12 20:48:26,444][43579] Avg episode reward: [(0, '277.370'), (1, '272.930')] [2023-10-12 20:48:27,410][44958] Updated weights for policy 0, policy_version 20130 (0.0008) [2023-10-12 20:48:27,782][44958] Updated weights for policy 0, policy_version 20140 (0.0007) [2023-10-12 20:48:28,159][44958] Updated weights for policy 0, policy_version 20150 (0.0007) [2023-10-12 20:48:28,539][44958] Updated weights for policy 0, policy_version 20160 (0.0007) [2023-10-12 20:48:29,606][44959] Updated weights for policy 1, policy_version 20260 (0.0010) [2023-10-12 20:48:29,974][44959] Updated weights for policy 1, policy_version 20270 (0.0008) [2023-10-12 20:48:30,344][44959] Updated weights for policy 1, policy_version 20280 (0.0007) [2023-10-12 20:48:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41418752. Throughput: 0: 1637.3, 1: 1654.8. Samples: 10364852. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-12 20:48:31,443][43579] Avg episode reward: [(0, '265.430'), (1, '266.180')] [2023-10-12 20:48:32,561][44958] Updated weights for policy 0, policy_version 20170 (0.0007) [2023-10-12 20:48:32,940][44958] Updated weights for policy 0, policy_version 20180 (0.0007) [2023-10-12 20:48:33,312][44958] Updated weights for policy 0, policy_version 20190 (0.0009) [2023-10-12 20:48:34,422][44959] Updated weights for policy 1, policy_version 20290 (0.0009) [2023-10-12 20:48:34,785][44959] Updated weights for policy 1, policy_version 20300 (0.0009) [2023-10-12 20:48:35,148][44959] Updated weights for policy 1, policy_version 20310 (0.0008) [2023-10-12 20:48:35,511][44959] Updated weights for policy 1, policy_version 20320 (0.0010) [2023-10-12 20:48:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41484288. Throughput: 0: 1638.1, 1: 1650.5. Samples: 10374816. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-12 20:48:36,443][43579] Avg episode reward: [(0, '265.870'), (1, '263.290')] [2023-10-12 20:48:37,400][44958] Updated weights for policy 0, policy_version 20200 (0.0009) [2023-10-12 20:48:37,784][44958] Updated weights for policy 0, policy_version 20210 (0.0009) [2023-10-12 20:48:38,164][44958] Updated weights for policy 0, policy_version 20220 (0.0007) [2023-10-12 20:48:39,744][44959] Updated weights for policy 1, policy_version 20330 (0.0009) [2023-10-12 20:48:40,117][44959] Updated weights for policy 1, policy_version 20340 (0.0010) [2023-10-12 20:48:40,478][44959] Updated weights for policy 1, policy_version 20350 (0.0009) [2023-10-12 20:48:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41549824. Throughput: 0: 1637.4, 1: 1639.7. Samples: 10394514. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-12 20:48:41,444][43579] Avg episode reward: [(0, '267.980'), (1, '267.790')] [2023-10-12 20:48:42,343][44958] Updated weights for policy 0, policy_version 20230 (0.0007) [2023-10-12 20:48:42,737][44958] Updated weights for policy 0, policy_version 20240 (0.0009) [2023-10-12 20:48:43,109][44958] Updated weights for policy 0, policy_version 20250 (0.0008) [2023-10-12 20:48:44,813][44959] Updated weights for policy 1, policy_version 20360 (0.0008) [2023-10-12 20:48:45,197][44959] Updated weights for policy 1, policy_version 20370 (0.0009) [2023-10-12 20:48:45,564][44959] Updated weights for policy 1, policy_version 20380 (0.0008) [2023-10-12 20:48:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41615360. Throughput: 0: 1639.0, 1: 1643.1. Samples: 10413860. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-12 20:48:46,444][43579] Avg episode reward: [(0, '267.950'), (1, '265.070')] [2023-10-12 20:48:47,400][44958] Updated weights for policy 0, policy_version 20260 (0.0009) [2023-10-12 20:48:47,773][44958] Updated weights for policy 0, policy_version 20270 (0.0009) [2023-10-12 20:48:48,154][44958] Updated weights for policy 0, policy_version 20280 (0.0008) [2023-10-12 20:48:49,835][44959] Updated weights for policy 1, policy_version 20390 (0.0010) [2023-10-12 20:48:50,201][44959] Updated weights for policy 1, policy_version 20400 (0.0008) [2023-10-12 20:48:50,570][44959] Updated weights for policy 1, policy_version 20410 (0.0008) [2023-10-12 20:48:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41680896. Throughput: 0: 1639.5, 1: 1636.0. Samples: 10423818. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-12 20:48:51,444][43579] Avg episode reward: [(0, '270.200'), (1, '268.240')] [2023-10-12 20:48:52,308][44958] Updated weights for policy 0, policy_version 20290 (0.0010) [2023-10-12 20:48:52,669][44958] Updated weights for policy 0, policy_version 20300 (0.0007) [2023-10-12 20:48:53,046][44958] Updated weights for policy 0, policy_version 20310 (0.0007) [2023-10-12 20:48:53,416][44958] Updated weights for policy 0, policy_version 20320 (0.0010) [2023-10-12 20:48:54,716][44959] Updated weights for policy 1, policy_version 20420 (0.0011) [2023-10-12 20:48:55,095][44959] Updated weights for policy 1, policy_version 20430 (0.0011) [2023-10-12 20:48:55,454][44959] Updated weights for policy 1, policy_version 20440 (0.0008) [2023-10-12 20:48:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41746432. Throughput: 0: 1642.6, 1: 1641.6. Samples: 10443868. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-12 20:48:56,444][43579] Avg episode reward: [(0, '268.820'), (1, '265.130')] [2023-10-12 20:48:57,503][44958] Updated weights for policy 0, policy_version 20330 (0.0011) [2023-10-12 20:48:57,870][44958] Updated weights for policy 0, policy_version 20340 (0.0009) [2023-10-12 20:48:58,255][44958] Updated weights for policy 0, policy_version 20350 (0.0008) [2023-10-12 20:48:59,640][44959] Updated weights for policy 1, policy_version 20450 (0.0008) [2023-10-12 20:49:00,009][44959] Updated weights for policy 1, policy_version 20460 (0.0007) [2023-10-12 20:49:00,383][44959] Updated weights for policy 1, policy_version 20470 (0.0007) [2023-10-12 20:49:00,754][44959] Updated weights for policy 1, policy_version 20480 (0.0007) [2023-10-12 20:49:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41811968. Throughput: 0: 1646.0, 1: 1640.4. Samples: 10463500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:49:01,444][43579] Avg episode reward: [(0, '276.990'), (1, '269.360')] [2023-10-12 20:49:02,152][44958] Updated weights for policy 0, policy_version 20360 (0.0008) [2023-10-12 20:49:02,521][44958] Updated weights for policy 0, policy_version 20370 (0.0008) [2023-10-12 20:49:02,907][44958] Updated weights for policy 0, policy_version 20380 (0.0008) [2023-10-12 20:49:04,851][44959] Updated weights for policy 1, policy_version 20490 (0.0011) [2023-10-12 20:49:05,216][44959] Updated weights for policy 1, policy_version 20500 (0.0010) [2023-10-12 20:49:05,579][44959] Updated weights for policy 1, policy_version 20510 (0.0010) [2023-10-12 20:49:06,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41877504. Throughput: 0: 1646.6, 1: 1643.9. Samples: 10473488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:49:06,443][43579] Avg episode reward: [(0, '274.460'), (1, '270.350')] [2023-10-12 20:49:07,271][44958] Updated weights for policy 0, policy_version 20390 (0.0009) [2023-10-12 20:49:07,650][44958] Updated weights for policy 0, policy_version 20400 (0.0010) [2023-10-12 20:49:08,023][44958] Updated weights for policy 0, policy_version 20410 (0.0007) [2023-10-12 20:49:09,818][44959] Updated weights for policy 1, policy_version 20520 (0.0008) [2023-10-12 20:49:10,192][44959] Updated weights for policy 1, policy_version 20530 (0.0009) [2023-10-12 20:49:10,554][44959] Updated weights for policy 1, policy_version 20540 (0.0010) [2023-10-12 20:49:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 41943040. Throughput: 0: 1638.9, 1: 1633.7. Samples: 10492764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:49:11,443][43579] Avg episode reward: [(0, '271.050'), (1, '270.630')] [2023-10-12 20:49:12,377][44958] Updated weights for policy 0, policy_version 20420 (0.0010) [2023-10-12 20:49:12,747][44958] Updated weights for policy 0, policy_version 20430 (0.0010) [2023-10-12 20:49:13,115][44958] Updated weights for policy 0, policy_version 20440 (0.0009) [2023-10-12 20:49:14,649][44959] Updated weights for policy 1, policy_version 20550 (0.0009) [2023-10-12 20:49:15,027][44959] Updated weights for policy 1, policy_version 20560 (0.0009) [2023-10-12 20:49:15,401][44959] Updated weights for policy 1, policy_version 20570 (0.0007) [2023-10-12 20:49:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42008576. Throughput: 0: 1645.5, 1: 1633.9. Samples: 10512430. Policy #0 lag: (min: 2.0, avg: 27.7, max: 32.0) [2023-10-12 20:49:16,444][43579] Avg episode reward: [(0, '270.500'), (1, '273.300')] [2023-10-12 20:49:17,204][44958] Updated weights for policy 0, policy_version 20450 (0.0008) [2023-10-12 20:49:17,577][44958] Updated weights for policy 0, policy_version 20460 (0.0009) [2023-10-12 20:49:17,945][44958] Updated weights for policy 0, policy_version 20470 (0.0008) [2023-10-12 20:49:18,318][44958] Updated weights for policy 0, policy_version 20480 (0.0009) [2023-10-12 20:49:19,605][44959] Updated weights for policy 1, policy_version 20580 (0.0009) [2023-10-12 20:49:19,973][44959] Updated weights for policy 1, policy_version 20590 (0.0010) [2023-10-12 20:49:20,335][44959] Updated weights for policy 1, policy_version 20600 (0.0009) [2023-10-12 20:49:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 42074112. Throughput: 0: 1642.2, 1: 1637.7. Samples: 10522412. Policy #0 lag: (min: 2.0, avg: 27.7, max: 32.0) [2023-10-12 20:49:21,444][43579] Avg episode reward: [(0, '270.700'), (1, '274.300')] [2023-10-12 20:49:22,343][44958] Updated weights for policy 0, policy_version 20490 (0.0009) [2023-10-12 20:49:22,705][44958] Updated weights for policy 0, policy_version 20500 (0.0011) [2023-10-12 20:49:23,090][44958] Updated weights for policy 0, policy_version 20510 (0.0009) [2023-10-12 20:49:24,369][44959] Updated weights for policy 1, policy_version 20610 (0.0009) [2023-10-12 20:49:24,731][44959] Updated weights for policy 1, policy_version 20620 (0.0009) [2023-10-12 20:49:25,102][44959] Updated weights for policy 1, policy_version 20630 (0.0008) [2023-10-12 20:49:25,477][44959] Updated weights for policy 1, policy_version 20640 (0.0008) [2023-10-12 20:49:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42139648. Throughput: 0: 1640.0, 1: 1642.4. Samples: 10542222. Policy #0 lag: (min: 2.0, avg: 27.7, max: 32.0) [2023-10-12 20:49:26,444][43579] Avg episode reward: [(0, '276.390'), (1, '277.610')] [2023-10-12 20:49:27,295][44958] Updated weights for policy 0, policy_version 20520 (0.0008) [2023-10-12 20:49:27,678][44958] Updated weights for policy 0, policy_version 20530 (0.0007) [2023-10-12 20:49:28,046][44958] Updated weights for policy 0, policy_version 20540 (0.0007) [2023-10-12 20:49:29,581][44959] Updated weights for policy 1, policy_version 20650 (0.0009) [2023-10-12 20:49:29,952][44959] Updated weights for policy 1, policy_version 20660 (0.0009) [2023-10-12 20:49:30,321][44959] Updated weights for policy 1, policy_version 20670 (0.0008) [2023-10-12 20:49:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42205184. Throughput: 0: 1648.0, 1: 1648.3. Samples: 10562194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-12 20:49:31,444][43579] Avg episode reward: [(0, '273.450'), (1, '280.230')] [2023-10-12 20:49:32,060][44958] Updated weights for policy 0, policy_version 20550 (0.0009) [2023-10-12 20:49:32,436][44958] Updated weights for policy 0, policy_version 20560 (0.0009) [2023-10-12 20:49:32,798][44958] Updated weights for policy 0, policy_version 20570 (0.0007) [2023-10-12 20:49:34,588][44959] Updated weights for policy 1, policy_version 20680 (0.0008) [2023-10-12 20:49:34,966][44959] Updated weights for policy 1, policy_version 20690 (0.0008) [2023-10-12 20:49:35,341][44959] Updated weights for policy 1, policy_version 20700 (0.0009) [2023-10-12 20:49:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42270720. Throughput: 0: 1646.2, 1: 1648.4. Samples: 10572072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-12 20:49:36,443][43579] Avg episode reward: [(0, '275.700'), (1, '274.420')] [2023-10-12 20:49:36,948][44958] Updated weights for policy 0, policy_version 20580 (0.0010) [2023-10-12 20:49:37,322][44958] Updated weights for policy 0, policy_version 20590 (0.0009) [2023-10-12 20:49:37,689][44958] Updated weights for policy 0, policy_version 20600 (0.0010) [2023-10-12 20:49:39,656][44959] Updated weights for policy 1, policy_version 20710 (0.0010) [2023-10-12 20:49:40,020][44959] Updated weights for policy 1, policy_version 20720 (0.0010) [2023-10-12 20:49:40,391][44959] Updated weights for policy 1, policy_version 20730 (0.0008) [2023-10-12 20:49:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42336256. Throughput: 0: 1651.2, 1: 1633.4. Samples: 10591674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-12 20:49:41,444][43579] Avg episode reward: [(0, '274.790'), (1, '271.280')] [2023-10-12 20:49:41,675][44958] Updated weights for policy 0, policy_version 20610 (0.0008) [2023-10-12 20:49:42,041][44958] Updated weights for policy 0, policy_version 20620 (0.0011) [2023-10-12 20:49:42,413][44958] Updated weights for policy 0, policy_version 20630 (0.0008) [2023-10-12 20:49:42,790][44958] Updated weights for policy 0, policy_version 20640 (0.0010) [2023-10-12 20:49:44,468][44959] Updated weights for policy 1, policy_version 20740 (0.0008) [2023-10-12 20:49:44,846][44959] Updated weights for policy 1, policy_version 20750 (0.0010) [2023-10-12 20:49:45,203][44959] Updated weights for policy 1, policy_version 20760 (0.0011) [2023-10-12 20:49:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 42401792. Throughput: 0: 1653.2, 1: 1635.6. Samples: 10611494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-12 20:49:46,443][43579] Avg episode reward: [(0, '276.310'), (1, '276.260')] [2023-10-12 20:49:46,453][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000020640_21135360.pth... [2023-10-12 20:49:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000020768_21266432.pth... [2023-10-12 20:49:46,482][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000019104_19562496.pth [2023-10-12 20:49:46,489][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000019232_19693568.pth [2023-10-12 20:49:46,966][44958] Updated weights for policy 0, policy_version 20650 (0.0007) [2023-10-12 20:49:47,337][44958] Updated weights for policy 0, policy_version 20660 (0.0008) [2023-10-12 20:49:47,709][44958] Updated weights for policy 0, policy_version 20670 (0.0008) [2023-10-12 20:49:49,614][44959] Updated weights for policy 1, policy_version 20770 (0.0010) [2023-10-12 20:49:49,984][44959] Updated weights for policy 1, policy_version 20780 (0.0009) [2023-10-12 20:49:50,351][44959] Updated weights for policy 1, policy_version 20790 (0.0009) [2023-10-12 20:49:50,730][44959] Updated weights for policy 1, policy_version 20800 (0.0008) [2023-10-12 20:49:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42467328. Throughput: 0: 1654.8, 1: 1634.3. Samples: 10621502. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) [2023-10-12 20:49:51,444][43579] Avg episode reward: [(0, '274.060'), (1, '274.940')] [2023-10-12 20:49:51,766][44958] Updated weights for policy 0, policy_version 20680 (0.0008) [2023-10-12 20:49:52,147][44958] Updated weights for policy 0, policy_version 20690 (0.0007) [2023-10-12 20:49:52,523][44958] Updated weights for policy 0, policy_version 20700 (0.0010) [2023-10-12 20:49:54,851][44959] Updated weights for policy 1, policy_version 20810 (0.0009) [2023-10-12 20:49:55,232][44959] Updated weights for policy 1, policy_version 20820 (0.0008) [2023-10-12 20:49:55,596][44959] Updated weights for policy 1, policy_version 20830 (0.0008) [2023-10-12 20:49:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42532864. Throughput: 0: 1658.5, 1: 1637.6. Samples: 10641088. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) [2023-10-12 20:49:56,444][43579] Avg episode reward: [(0, '268.540'), (1, '270.190')] [2023-10-12 20:49:56,653][44958] Updated weights for policy 0, policy_version 20710 (0.0008) [2023-10-12 20:49:57,015][44958] Updated weights for policy 0, policy_version 20720 (0.0008) [2023-10-12 20:49:57,391][44958] Updated weights for policy 0, policy_version 20730 (0.0008) [2023-10-12 20:49:59,684][44959] Updated weights for policy 1, policy_version 20840 (0.0009) [2023-10-12 20:50:00,044][44959] Updated weights for policy 1, policy_version 20850 (0.0008) [2023-10-12 20:50:00,405][44959] Updated weights for policy 1, policy_version 20860 (0.0007) [2023-10-12 20:50:01,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 42598400. Throughput: 0: 1652.4, 1: 1644.3. Samples: 10660778. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) [2023-10-12 20:50:01,443][43579] Avg episode reward: [(0, '269.240'), (1, '273.020')] [2023-10-12 20:50:01,629][44958] Updated weights for policy 0, policy_version 20740 (0.0007) [2023-10-12 20:50:02,002][44958] Updated weights for policy 0, policy_version 20750 (0.0009) [2023-10-12 20:50:02,368][44958] Updated weights for policy 0, policy_version 20760 (0.0010) [2023-10-12 20:50:04,475][44959] Updated weights for policy 1, policy_version 20870 (0.0008) [2023-10-12 20:50:04,854][44959] Updated weights for policy 1, policy_version 20880 (0.0009) [2023-10-12 20:50:05,220][44959] Updated weights for policy 1, policy_version 20890 (0.0009) [2023-10-12 20:50:06,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42663936. Throughput: 0: 1655.0, 1: 1644.3. Samples: 10670880. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) [2023-10-12 20:50:06,443][43579] Avg episode reward: [(0, '269.720'), (1, '270.500')] [2023-10-12 20:50:06,525][44958] Updated weights for policy 0, policy_version 20770 (0.0009) [2023-10-12 20:50:06,901][44958] Updated weights for policy 0, policy_version 20780 (0.0009) [2023-10-12 20:50:07,271][44958] Updated weights for policy 0, policy_version 20790 (0.0009) [2023-10-12 20:50:07,646][44958] Updated weights for policy 0, policy_version 20800 (0.0009) [2023-10-12 20:50:09,450][44959] Updated weights for policy 1, policy_version 20900 (0.0008) [2023-10-12 20:50:09,822][44959] Updated weights for policy 1, policy_version 20910 (0.0009) [2023-10-12 20:50:10,183][44959] Updated weights for policy 1, policy_version 20920 (0.0010) [2023-10-12 20:50:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42729472. Throughput: 0: 1656.1, 1: 1633.4. Samples: 10690250. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:50:11,443][43579] Avg episode reward: [(0, '269.380'), (1, '265.930')] [2023-10-12 20:50:11,802][44958] Updated weights for policy 0, policy_version 20810 (0.0009) [2023-10-12 20:50:12,184][44958] Updated weights for policy 0, policy_version 20820 (0.0010) [2023-10-12 20:50:12,566][44958] Updated weights for policy 0, policy_version 20830 (0.0009) [2023-10-12 20:50:14,324][44959] Updated weights for policy 1, policy_version 20930 (0.0008) [2023-10-12 20:50:14,701][44959] Updated weights for policy 1, policy_version 20940 (0.0009) [2023-10-12 20:50:15,063][44959] Updated weights for policy 1, policy_version 20950 (0.0009) [2023-10-12 20:50:15,431][44959] Updated weights for policy 1, policy_version 20960 (0.0008) [2023-10-12 20:50:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42795008. Throughput: 0: 1652.5, 1: 1636.4. Samples: 10710194. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:50:16,443][43579] Avg episode reward: [(0, '268.720'), (1, '261.900')] [2023-10-12 20:50:16,777][44958] Updated weights for policy 0, policy_version 20840 (0.0008) [2023-10-12 20:50:17,146][44958] Updated weights for policy 0, policy_version 20850 (0.0007) [2023-10-12 20:50:17,520][44958] Updated weights for policy 0, policy_version 20860 (0.0007) [2023-10-12 20:50:19,613][44959] Updated weights for policy 1, policy_version 20970 (0.0009) [2023-10-12 20:50:20,000][44959] Updated weights for policy 1, policy_version 20980 (0.0009) [2023-10-12 20:50:20,371][44959] Updated weights for policy 1, policy_version 20990 (0.0009) [2023-10-12 20:50:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42860544. Throughput: 0: 1656.3, 1: 1640.8. Samples: 10720440. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:50:21,443][43579] Avg episode reward: [(0, '273.250'), (1, '261.550')] [2023-10-12 20:50:21,769][44958] Updated weights for policy 0, policy_version 20870 (0.0008) [2023-10-12 20:50:22,139][44958] Updated weights for policy 0, policy_version 20880 (0.0009) [2023-10-12 20:50:22,520][44958] Updated weights for policy 0, policy_version 20890 (0.0010) [2023-10-12 20:50:24,570][44959] Updated weights for policy 1, policy_version 21000 (0.0009) [2023-10-12 20:50:24,938][44959] Updated weights for policy 1, policy_version 21010 (0.0007) [2023-10-12 20:50:25,309][44959] Updated weights for policy 1, policy_version 21020 (0.0008) [2023-10-12 20:50:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42926080. Throughput: 0: 1649.2, 1: 1636.4. Samples: 10739530. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 20:50:26,444][43579] Avg episode reward: [(0, '273.680'), (1, '267.540')] [2023-10-12 20:50:26,788][44958] Updated weights for policy 0, policy_version 20900 (0.0009) [2023-10-12 20:50:27,168][44958] Updated weights for policy 0, policy_version 20910 (0.0010) [2023-10-12 20:50:27,539][44958] Updated weights for policy 0, policy_version 20920 (0.0009) [2023-10-12 20:50:29,461][44959] Updated weights for policy 1, policy_version 21030 (0.0008) [2023-10-12 20:50:29,832][44959] Updated weights for policy 1, policy_version 21040 (0.0009) [2023-10-12 20:50:30,216][44959] Updated weights for policy 1, policy_version 21050 (0.0008) [2023-10-12 20:50:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42991616. Throughput: 0: 1642.3, 1: 1639.5. Samples: 10759174. Policy #0 lag: (min: 9.0, avg: 19.6, max: 41.0) [2023-10-12 20:50:31,444][43579] Avg episode reward: [(0, '277.530'), (1, '268.190')] [2023-10-12 20:50:31,660][44958] Updated weights for policy 0, policy_version 20930 (0.0008) [2023-10-12 20:50:32,035][44958] Updated weights for policy 0, policy_version 20940 (0.0008) [2023-10-12 20:50:32,406][44958] Updated weights for policy 0, policy_version 20950 (0.0009) [2023-10-12 20:50:32,779][44958] Updated weights for policy 0, policy_version 20960 (0.0011) [2023-10-12 20:50:34,168][44959] Updated weights for policy 1, policy_version 21060 (0.0009) [2023-10-12 20:50:34,531][44959] Updated weights for policy 1, policy_version 21070 (0.0007) [2023-10-12 20:50:34,892][44959] Updated weights for policy 1, policy_version 21080 (0.0007) [2023-10-12 20:50:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43057152. Throughput: 0: 1642.3, 1: 1648.1. Samples: 10769572. Policy #0 lag: (min: 9.0, avg: 19.6, max: 41.0) [2023-10-12 20:50:36,444][43579] Avg episode reward: [(0, '275.860'), (1, '273.650')] [2023-10-12 20:50:36,854][44958] Updated weights for policy 0, policy_version 20970 (0.0008) [2023-10-12 20:50:37,224][44958] Updated weights for policy 0, policy_version 20980 (0.0008) [2023-10-12 20:50:37,593][44958] Updated weights for policy 0, policy_version 20990 (0.0008) [2023-10-12 20:50:39,153][44959] Updated weights for policy 1, policy_version 21090 (0.0008) [2023-10-12 20:50:39,514][44959] Updated weights for policy 1, policy_version 21100 (0.0008) [2023-10-12 20:50:39,878][44959] Updated weights for policy 1, policy_version 21110 (0.0010) [2023-10-12 20:50:40,252][44959] Updated weights for policy 1, policy_version 21120 (0.0008) [2023-10-12 20:50:41,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 43122688. Throughput: 0: 1643.3, 1: 1640.1. Samples: 10788844. Policy #0 lag: (min: 9.0, avg: 19.6, max: 41.0) [2023-10-12 20:50:41,443][43579] Avg episode reward: [(0, '273.890'), (1, '284.390')] [2023-10-12 20:50:41,444][44583] Saving new best policy, reward=284.390! [2023-10-12 20:50:41,888][44958] Updated weights for policy 0, policy_version 21000 (0.0009) [2023-10-12 20:50:42,257][44958] Updated weights for policy 0, policy_version 21010 (0.0009) [2023-10-12 20:50:42,637][44958] Updated weights for policy 0, policy_version 21020 (0.0007) [2023-10-12 20:50:44,586][44959] Updated weights for policy 1, policy_version 21130 (0.0010) [2023-10-12 20:50:44,953][44959] Updated weights for policy 1, policy_version 21140 (0.0008) [2023-10-12 20:50:45,328][44959] Updated weights for policy 1, policy_version 21150 (0.0008) [2023-10-12 20:50:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43188224. Throughput: 0: 1643.0, 1: 1645.2. Samples: 10808748. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-12 20:50:46,443][43579] Avg episode reward: [(0, '274.890'), (1, '285.220')] [2023-10-12 20:50:46,455][44583] Saving new best policy, reward=285.220! [2023-10-12 20:50:46,688][44958] Updated weights for policy 0, policy_version 21030 (0.0008) [2023-10-12 20:50:47,056][44958] Updated weights for policy 0, policy_version 21040 (0.0008) [2023-10-12 20:50:47,425][44958] Updated weights for policy 0, policy_version 21050 (0.0008) [2023-10-12 20:50:49,502][44959] Updated weights for policy 1, policy_version 21160 (0.0007) [2023-10-12 20:50:49,873][44959] Updated weights for policy 1, policy_version 21170 (0.0008) [2023-10-12 20:50:50,243][44959] Updated weights for policy 1, policy_version 21180 (0.0011) [2023-10-12 20:50:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 43253760. Throughput: 0: 1640.4, 1: 1644.4. Samples: 10818692. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-12 20:50:51,443][43579] Avg episode reward: [(0, '268.050'), (1, '287.120')] [2023-10-12 20:50:51,444][44583] Saving new best policy, reward=287.120! [2023-10-12 20:50:51,911][44958] Updated weights for policy 0, policy_version 21060 (0.0008) [2023-10-12 20:50:52,300][44958] Updated weights for policy 0, policy_version 21070 (0.0008) [2023-10-12 20:50:52,666][44958] Updated weights for policy 0, policy_version 21080 (0.0008) [2023-10-12 20:50:54,523][44959] Updated weights for policy 1, policy_version 21190 (0.0008) [2023-10-12 20:50:54,898][44959] Updated weights for policy 1, policy_version 21200 (0.0007) [2023-10-12 20:50:55,273][44959] Updated weights for policy 1, policy_version 21210 (0.0008) [2023-10-12 20:50:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43319296. Throughput: 0: 1640.8, 1: 1646.4. Samples: 10838174. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-12 20:50:56,443][43579] Avg episode reward: [(0, '266.730'), (1, '282.880')] [2023-10-12 20:50:56,724][44958] Updated weights for policy 0, policy_version 21090 (0.0008) [2023-10-12 20:50:57,098][44958] Updated weights for policy 0, policy_version 21100 (0.0007) [2023-10-12 20:50:57,471][44958] Updated weights for policy 0, policy_version 21110 (0.0007) [2023-10-12 20:50:57,839][44958] Updated weights for policy 0, policy_version 21120 (0.0007) [2023-10-12 20:50:59,314][44959] Updated weights for policy 1, policy_version 21220 (0.0008) [2023-10-12 20:50:59,683][44959] Updated weights for policy 1, policy_version 21230 (0.0009) [2023-10-12 20:51:00,051][44959] Updated weights for policy 1, policy_version 21240 (0.0010) [2023-10-12 20:51:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43384832. Throughput: 0: 1638.6, 1: 1646.6. Samples: 10858026. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-12 20:51:01,443][43579] Avg episode reward: [(0, '261.140'), (1, '278.140')] [2023-10-12 20:51:02,107][44958] Updated weights for policy 0, policy_version 21130 (0.0010) [2023-10-12 20:51:02,475][44958] Updated weights for policy 0, policy_version 21140 (0.0008) [2023-10-12 20:51:02,858][44958] Updated weights for policy 0, policy_version 21150 (0.0007) [2023-10-12 20:51:04,133][44959] Updated weights for policy 1, policy_version 21250 (0.0007) [2023-10-12 20:51:04,560][44959] Updated weights for policy 1, policy_version 21260 (0.0007) [2023-10-12 20:51:04,928][44959] Updated weights for policy 1, policy_version 21270 (0.0007) [2023-10-12 20:51:05,294][44959] Updated weights for policy 1, policy_version 21280 (0.0010) [2023-10-12 20:51:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43450368. Throughput: 0: 1632.7, 1: 1645.5. Samples: 10867958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:51:06,443][43579] Avg episode reward: [(0, '264.780'), (1, '269.490')] [2023-10-12 20:51:07,021][44958] Updated weights for policy 0, policy_version 21160 (0.0008) [2023-10-12 20:51:07,395][44958] Updated weights for policy 0, policy_version 21170 (0.0008) [2023-10-12 20:51:07,776][44958] Updated weights for policy 0, policy_version 21180 (0.0007) [2023-10-12 20:51:09,459][44959] Updated weights for policy 1, policy_version 21290 (0.0010) [2023-10-12 20:51:09,818][44959] Updated weights for policy 1, policy_version 21300 (0.0010) [2023-10-12 20:51:10,188][44959] Updated weights for policy 1, policy_version 21310 (0.0010) [2023-10-12 20:51:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43515904. Throughput: 0: 1638.3, 1: 1644.8. Samples: 10887270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:51:11,443][43579] Avg episode reward: [(0, '265.460'), (1, '266.570')] [2023-10-12 20:51:11,750][44958] Updated weights for policy 0, policy_version 21190 (0.0007) [2023-10-12 20:51:12,127][44958] Updated weights for policy 0, policy_version 21200 (0.0010) [2023-10-12 20:51:12,493][44958] Updated weights for policy 0, policy_version 21210 (0.0011) [2023-10-12 20:51:14,439][44959] Updated weights for policy 1, policy_version 21320 (0.0008) [2023-10-12 20:51:14,806][44959] Updated weights for policy 1, policy_version 21330 (0.0007) [2023-10-12 20:51:15,177][44959] Updated weights for policy 1, policy_version 21340 (0.0009) [2023-10-12 20:51:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43581440. Throughput: 0: 1644.0, 1: 1650.9. Samples: 10907444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:51:16,443][43579] Avg episode reward: [(0, '261.250'), (1, '265.370')] [2023-10-12 20:51:16,781][44958] Updated weights for policy 0, policy_version 21220 (0.0009) [2023-10-12 20:51:17,147][44958] Updated weights for policy 0, policy_version 21230 (0.0008) [2023-10-12 20:51:17,514][44958] Updated weights for policy 0, policy_version 21240 (0.0007) [2023-10-12 20:51:19,203][44959] Updated weights for policy 1, policy_version 21350 (0.0007) [2023-10-12 20:51:19,566][44959] Updated weights for policy 1, policy_version 21360 (0.0010) [2023-10-12 20:51:19,935][44959] Updated weights for policy 1, policy_version 21370 (0.0010) [2023-10-12 20:51:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43646976. Throughput: 0: 1641.9, 1: 1642.0. Samples: 10917348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:51:21,444][43579] Avg episode reward: [(0, '262.960'), (1, '260.970')] [2023-10-12 20:51:21,849][44958] Updated weights for policy 0, policy_version 21250 (0.0007) [2023-10-12 20:51:22,222][44958] Updated weights for policy 0, policy_version 21260 (0.0008) [2023-10-12 20:51:22,595][44958] Updated weights for policy 0, policy_version 21270 (0.0009) [2023-10-12 20:51:22,971][44958] Updated weights for policy 0, policy_version 21280 (0.0009) [2023-10-12 20:51:24,193][44959] Updated weights for policy 1, policy_version 21380 (0.0009) [2023-10-12 20:51:24,562][44959] Updated weights for policy 1, policy_version 21390 (0.0007) [2023-10-12 20:51:24,931][44959] Updated weights for policy 1, policy_version 21400 (0.0011) [2023-10-12 20:51:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43712512. Throughput: 0: 1641.4, 1: 1645.2. Samples: 10936742. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) [2023-10-12 20:51:26,443][43579] Avg episode reward: [(0, '268.480'), (1, '262.940')] [2023-10-12 20:51:27,034][44958] Updated weights for policy 0, policy_version 21290 (0.0009) [2023-10-12 20:51:27,400][44958] Updated weights for policy 0, policy_version 21300 (0.0010) [2023-10-12 20:51:27,781][44958] Updated weights for policy 0, policy_version 21310 (0.0010) [2023-10-12 20:51:29,130][44959] Updated weights for policy 1, policy_version 21410 (0.0011) [2023-10-12 20:51:29,485][44959] Updated weights for policy 1, policy_version 21420 (0.0009) [2023-10-12 20:51:29,864][44959] Updated weights for policy 1, policy_version 21430 (0.0009) [2023-10-12 20:51:30,235][44959] Updated weights for policy 1, policy_version 21440 (0.0009) [2023-10-12 20:51:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43778048. Throughput: 0: 1639.6, 1: 1642.8. Samples: 10956458. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) [2023-10-12 20:51:31,444][43579] Avg episode reward: [(0, '267.270'), (1, '268.450')] [2023-10-12 20:51:31,777][44958] Updated weights for policy 0, policy_version 21320 (0.0007) [2023-10-12 20:51:32,149][44958] Updated weights for policy 0, policy_version 21330 (0.0007) [2023-10-12 20:51:32,519][44958] Updated weights for policy 0, policy_version 21340 (0.0008) [2023-10-12 20:51:34,443][44959] Updated weights for policy 1, policy_version 21450 (0.0011) [2023-10-12 20:51:34,821][44959] Updated weights for policy 1, policy_version 21460 (0.0008) [2023-10-12 20:51:35,181][44959] Updated weights for policy 1, policy_version 21470 (0.0008) [2023-10-12 20:51:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 43843584. Throughput: 0: 1642.6, 1: 1645.9. Samples: 10966672. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) [2023-10-12 20:51:36,443][43579] Avg episode reward: [(0, '269.840'), (1, '272.100')] [2023-10-12 20:51:36,653][44958] Updated weights for policy 0, policy_version 21350 (0.0012) [2023-10-12 20:51:37,026][44958] Updated weights for policy 0, policy_version 21360 (0.0008) [2023-10-12 20:51:37,397][44958] Updated weights for policy 0, policy_version 21370 (0.0011) [2023-10-12 20:51:39,232][44959] Updated weights for policy 1, policy_version 21480 (0.0007) [2023-10-12 20:51:39,600][44959] Updated weights for policy 1, policy_version 21490 (0.0007) [2023-10-12 20:51:39,964][44959] Updated weights for policy 1, policy_version 21500 (0.0009) [2023-10-12 20:51:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 43909120. Throughput: 0: 1646.4, 1: 1644.3. Samples: 10986256. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) [2023-10-12 20:51:41,444][43579] Avg episode reward: [(0, '270.850'), (1, '270.990')] [2023-10-12 20:51:41,579][44958] Updated weights for policy 0, policy_version 21380 (0.0008) [2023-10-12 20:51:41,953][44958] Updated weights for policy 0, policy_version 21390 (0.0007) [2023-10-12 20:51:42,330][44958] Updated weights for policy 0, policy_version 21400 (0.0008) [2023-10-12 20:51:44,089][44959] Updated weights for policy 1, policy_version 21510 (0.0009) [2023-10-12 20:51:44,460][44959] Updated weights for policy 1, policy_version 21520 (0.0007) [2023-10-12 20:51:44,824][44959] Updated weights for policy 1, policy_version 21530 (0.0008) [2023-10-12 20:51:46,370][44958] Updated weights for policy 0, policy_version 21410 (0.0008) [2023-10-12 20:51:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 43974656. Throughput: 0: 1647.7, 1: 1651.4. Samples: 11006488. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-12 20:51:46,443][43579] Avg episode reward: [(0, '279.990'), (1, '270.600')] [2023-10-12 20:51:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000021536_22052864.pth... [2023-10-12 20:51:46,488][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000020000_20480000.pth [2023-10-12 20:51:46,768][44958] Updated weights for policy 0, policy_version 21420 (0.0011) [2023-10-12 20:51:47,151][44958] Updated weights for policy 0, policy_version 21430 (0.0008) [2023-10-12 20:51:47,516][44958] Updated weights for policy 0, policy_version 21440 (0.0009) [2023-10-12 20:51:47,516][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000021440_21954560.pth... [2023-10-12 20:51:47,558][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000019872_20348928.pth [2023-10-12 20:51:49,093][44959] Updated weights for policy 1, policy_version 21540 (0.0007) [2023-10-12 20:51:49,501][44959] Updated weights for policy 1, policy_version 21550 (0.0007) [2023-10-12 20:51:49,872][44959] Updated weights for policy 1, policy_version 21560 (0.0009) [2023-10-12 20:51:51,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 44040192. Throughput: 0: 1647.9, 1: 1642.2. Samples: 11016012. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-12 20:51:51,443][43579] Avg episode reward: [(0, '276.430'), (1, '278.580')] [2023-10-12 20:51:51,688][44958] Updated weights for policy 0, policy_version 21450 (0.0008) [2023-10-12 20:51:52,066][44958] Updated weights for policy 0, policy_version 21460 (0.0010) [2023-10-12 20:51:52,437][44958] Updated weights for policy 0, policy_version 21470 (0.0007) [2023-10-12 20:51:53,987][44959] Updated weights for policy 1, policy_version 21570 (0.0008) [2023-10-12 20:51:54,360][44959] Updated weights for policy 1, policy_version 21580 (0.0008) [2023-10-12 20:51:54,727][44959] Updated weights for policy 1, policy_version 21590 (0.0008) [2023-10-12 20:51:55,095][44959] Updated weights for policy 1, policy_version 21600 (0.0007) [2023-10-12 20:51:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 44105728. Throughput: 0: 1646.4, 1: 1642.2. Samples: 11035256. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-12 20:51:56,443][43579] Avg episode reward: [(0, '274.570'), (1, '279.640')] [2023-10-12 20:51:56,606][44958] Updated weights for policy 0, policy_version 21480 (0.0009) [2023-10-12 20:51:56,981][44958] Updated weights for policy 0, policy_version 21490 (0.0010) [2023-10-12 20:51:57,354][44958] Updated weights for policy 0, policy_version 21500 (0.0008) [2023-10-12 20:51:59,073][44959] Updated weights for policy 1, policy_version 21610 (0.0009) [2023-10-12 20:51:59,444][44959] Updated weights for policy 1, policy_version 21620 (0.0010) [2023-10-12 20:51:59,816][44959] Updated weights for policy 1, policy_version 21630 (0.0008) [2023-10-12 20:52:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 44171264. Throughput: 0: 1638.8, 1: 1652.4. Samples: 11055548. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-12 20:52:01,443][43579] Avg episode reward: [(0, '274.990'), (1, '281.000')] [2023-10-12 20:52:01,569][44958] Updated weights for policy 0, policy_version 21510 (0.0010) [2023-10-12 20:52:01,947][44958] Updated weights for policy 0, policy_version 21520 (0.0008) [2023-10-12 20:52:02,318][44958] Updated weights for policy 0, policy_version 21530 (0.0007) [2023-10-12 20:52:03,966][44959] Updated weights for policy 1, policy_version 21640 (0.0007) [2023-10-12 20:52:04,344][44959] Updated weights for policy 1, policy_version 21650 (0.0007) [2023-10-12 20:52:04,717][44959] Updated weights for policy 1, policy_version 21660 (0.0008) [2023-10-12 20:52:06,426][44958] Updated weights for policy 0, policy_version 21540 (0.0008) [2023-10-12 20:52:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44236800. Throughput: 0: 1640.1, 1: 1647.3. Samples: 11065280. Policy #0 lag: (min: 10.0, avg: 16.9, max: 42.0) [2023-10-12 20:52:06,443][43579] Avg episode reward: [(0, '276.390'), (1, '279.430')] [2023-10-12 20:52:06,810][44958] Updated weights for policy 0, policy_version 21550 (0.0007) [2023-10-12 20:52:07,190][44958] Updated weights for policy 0, policy_version 21560 (0.0008) [2023-10-12 20:52:08,893][44959] Updated weights for policy 1, policy_version 21670 (0.0007) [2023-10-12 20:52:09,258][44959] Updated weights for policy 1, policy_version 21680 (0.0008) [2023-10-12 20:52:09,633][44959] Updated weights for policy 1, policy_version 21690 (0.0008) [2023-10-12 20:52:11,350][44958] Updated weights for policy 0, policy_version 21570 (0.0009) [2023-10-12 20:52:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44302336. Throughput: 0: 1645.2, 1: 1648.0. Samples: 11084938. Policy #0 lag: (min: 10.0, avg: 16.9, max: 42.0) [2023-10-12 20:52:11,443][43579] Avg episode reward: [(0, '274.310'), (1, '277.880')] [2023-10-12 20:52:11,721][44958] Updated weights for policy 0, policy_version 21580 (0.0009) [2023-10-12 20:52:12,088][44958] Updated weights for policy 0, policy_version 21590 (0.0009) [2023-10-12 20:52:12,468][44958] Updated weights for policy 0, policy_version 21600 (0.0010) [2023-10-12 20:52:13,820][44959] Updated weights for policy 1, policy_version 21700 (0.0008) [2023-10-12 20:52:14,200][44959] Updated weights for policy 1, policy_version 21710 (0.0007) [2023-10-12 20:52:14,576][44959] Updated weights for policy 1, policy_version 21720 (0.0008) [2023-10-12 20:52:16,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 44367872. Throughput: 0: 1648.0, 1: 1658.2. Samples: 11105234. Policy #0 lag: (min: 10.0, avg: 16.9, max: 42.0) [2023-10-12 20:52:16,444][43579] Avg episode reward: [(0, '272.160'), (1, '271.340')] [2023-10-12 20:52:16,488][44958] Updated weights for policy 0, policy_version 21610 (0.0009) [2023-10-12 20:52:16,866][44958] Updated weights for policy 0, policy_version 21620 (0.0007) [2023-10-12 20:52:17,233][44958] Updated weights for policy 0, policy_version 21630 (0.0010) [2023-10-12 20:52:18,697][44959] Updated weights for policy 1, policy_version 21730 (0.0008) [2023-10-12 20:52:19,059][44959] Updated weights for policy 1, policy_version 21740 (0.0009) [2023-10-12 20:52:19,429][44959] Updated weights for policy 1, policy_version 21750 (0.0007) [2023-10-12 20:52:19,797][44959] Updated weights for policy 1, policy_version 21760 (0.0008) [2023-10-12 20:52:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44433408. Throughput: 0: 1645.9, 1: 1646.5. Samples: 11114832. Policy #0 lag: (min: 10.0, avg: 16.9, max: 42.0) [2023-10-12 20:52:21,443][43579] Avg episode reward: [(0, '275.440'), (1, '267.820')] [2023-10-12 20:52:21,522][44958] Updated weights for policy 0, policy_version 21640 (0.0011) [2023-10-12 20:52:21,890][44958] Updated weights for policy 0, policy_version 21650 (0.0009) [2023-10-12 20:52:22,269][44958] Updated weights for policy 0, policy_version 21660 (0.0007) [2023-10-12 20:52:23,960][44959] Updated weights for policy 1, policy_version 21770 (0.0008) [2023-10-12 20:52:24,332][44959] Updated weights for policy 1, policy_version 21780 (0.0008) [2023-10-12 20:52:24,710][44959] Updated weights for policy 1, policy_version 21790 (0.0010) [2023-10-12 20:52:26,233][44958] Updated weights for policy 0, policy_version 21670 (0.0009) [2023-10-12 20:52:26,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44498944. Throughput: 0: 1644.4, 1: 1653.3. Samples: 11134650. Policy #0 lag: (min: 28.0, avg: 33.7, max: 60.0) [2023-10-12 20:52:26,443][43579] Avg episode reward: [(0, '272.840'), (1, '263.670')] [2023-10-12 20:52:26,607][44958] Updated weights for policy 0, policy_version 21680 (0.0010) [2023-10-12 20:52:26,975][44958] Updated weights for policy 0, policy_version 21690 (0.0008) [2023-10-12 20:52:28,963][44959] Updated weights for policy 1, policy_version 21800 (0.0010) [2023-10-12 20:52:29,326][44959] Updated weights for policy 1, policy_version 21810 (0.0011) [2023-10-12 20:52:29,690][44959] Updated weights for policy 1, policy_version 21820 (0.0010) [2023-10-12 20:52:31,297][44958] Updated weights for policy 0, policy_version 21700 (0.0009) [2023-10-12 20:52:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44564480. Throughput: 0: 1641.1, 1: 1652.0. Samples: 11154678. Policy #0 lag: (min: 28.0, avg: 33.7, max: 60.0) [2023-10-12 20:52:31,444][43579] Avg episode reward: [(0, '276.550'), (1, '258.910')] [2023-10-12 20:52:31,694][44958] Updated weights for policy 0, policy_version 21710 (0.0008) [2023-10-12 20:52:32,065][44958] Updated weights for policy 0, policy_version 21720 (0.0009) [2023-10-12 20:52:33,613][44959] Updated weights for policy 1, policy_version 21830 (0.0009) [2023-10-12 20:52:33,998][44959] Updated weights for policy 1, policy_version 21840 (0.0008) [2023-10-12 20:52:34,367][44959] Updated weights for policy 1, policy_version 21850 (0.0009) [2023-10-12 20:52:36,110][44958] Updated weights for policy 0, policy_version 21730 (0.0008) [2023-10-12 20:52:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44630016. Throughput: 0: 1647.0, 1: 1646.0. Samples: 11164196. Policy #0 lag: (min: 28.0, avg: 33.7, max: 60.0) [2023-10-12 20:52:36,443][43579] Avg episode reward: [(0, '274.740'), (1, '261.320')] [2023-10-12 20:52:36,498][44958] Updated weights for policy 0, policy_version 21740 (0.0009) [2023-10-12 20:52:36,874][44958] Updated weights for policy 0, policy_version 21750 (0.0008) [2023-10-12 20:52:37,246][44958] Updated weights for policy 0, policy_version 21760 (0.0008) [2023-10-12 20:52:38,523][44959] Updated weights for policy 1, policy_version 21860 (0.0009) [2023-10-12 20:52:38,892][44959] Updated weights for policy 1, policy_version 21870 (0.0007) [2023-10-12 20:52:39,257][44959] Updated weights for policy 1, policy_version 21880 (0.0008) [2023-10-12 20:52:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44695552. Throughput: 0: 1646.0, 1: 1658.9. Samples: 11183976. Policy #0 lag: (min: 28.0, avg: 33.7, max: 60.0) [2023-10-12 20:52:41,444][43579] Avg episode reward: [(0, '272.370'), (1, '263.230')] [2023-10-12 20:52:41,446][44958] Updated weights for policy 0, policy_version 21770 (0.0011) [2023-10-12 20:52:41,816][44958] Updated weights for policy 0, policy_version 21780 (0.0010) [2023-10-12 20:52:42,199][44958] Updated weights for policy 0, policy_version 21790 (0.0008) [2023-10-12 20:52:43,424][44959] Updated weights for policy 1, policy_version 21890 (0.0008) [2023-10-12 20:52:43,790][44959] Updated weights for policy 1, policy_version 21900 (0.0009) [2023-10-12 20:52:44,166][44959] Updated weights for policy 1, policy_version 21910 (0.0007) [2023-10-12 20:52:44,525][44959] Updated weights for policy 1, policy_version 21920 (0.0009) [2023-10-12 20:52:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44761088. Throughput: 0: 1643.1, 1: 1658.9. Samples: 11204140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:52:46,443][43579] Avg episode reward: [(0, '270.260'), (1, '266.830')] [2023-10-12 20:52:46,559][44958] Updated weights for policy 0, policy_version 21800 (0.0007) [2023-10-12 20:52:46,931][44958] Updated weights for policy 0, policy_version 21810 (0.0008) [2023-10-12 20:52:47,300][44958] Updated weights for policy 0, policy_version 21820 (0.0009) [2023-10-12 20:52:48,605][44959] Updated weights for policy 1, policy_version 21930 (0.0008) [2023-10-12 20:52:48,971][44959] Updated weights for policy 1, policy_version 21940 (0.0008) [2023-10-12 20:52:49,338][44959] Updated weights for policy 1, policy_version 21950 (0.0008) [2023-10-12 20:52:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44826624. Throughput: 0: 1643.2, 1: 1648.6. Samples: 11213412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:52:51,443][43579] Avg episode reward: [(0, '270.610'), (1, '272.520')] [2023-10-12 20:52:51,455][44958] Updated weights for policy 0, policy_version 21830 (0.0008) [2023-10-12 20:52:51,839][44958] Updated weights for policy 0, policy_version 21840 (0.0008) [2023-10-12 20:52:52,207][44958] Updated weights for policy 0, policy_version 21850 (0.0007) [2023-10-12 20:52:53,550][44959] Updated weights for policy 1, policy_version 21960 (0.0009) [2023-10-12 20:52:53,916][44959] Updated weights for policy 1, policy_version 21970 (0.0010) [2023-10-12 20:52:54,290][44959] Updated weights for policy 1, policy_version 21980 (0.0007) [2023-10-12 20:52:56,438][44958] Updated weights for policy 0, policy_version 21860 (0.0009) [2023-10-12 20:52:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44892160. Throughput: 0: 1642.1, 1: 1652.3. Samples: 11233188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:52:56,443][43579] Avg episode reward: [(0, '270.410'), (1, '276.340')] [2023-10-12 20:52:56,811][44958] Updated weights for policy 0, policy_version 21870 (0.0008) [2023-10-12 20:52:57,180][44958] Updated weights for policy 0, policy_version 21880 (0.0008) [2023-10-12 20:52:58,470][44959] Updated weights for policy 1, policy_version 21990 (0.0008) [2023-10-12 20:52:58,841][44959] Updated weights for policy 1, policy_version 22000 (0.0008) [2023-10-12 20:52:59,199][44959] Updated weights for policy 1, policy_version 22010 (0.0008) [2023-10-12 20:53:01,273][44958] Updated weights for policy 0, policy_version 21890 (0.0007) [2023-10-12 20:53:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 44957696. Throughput: 0: 1641.2, 1: 1656.0. Samples: 11253606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:53:01,443][43579] Avg episode reward: [(0, '267.710'), (1, '279.610')] [2023-10-12 20:53:01,653][44958] Updated weights for policy 0, policy_version 21900 (0.0008) [2023-10-12 20:53:02,026][44958] Updated weights for policy 0, policy_version 21910 (0.0008) [2023-10-12 20:53:02,396][44958] Updated weights for policy 0, policy_version 21920 (0.0007) [2023-10-12 20:53:03,159][44959] Updated weights for policy 1, policy_version 22020 (0.0007) [2023-10-12 20:53:03,526][44959] Updated weights for policy 1, policy_version 22030 (0.0008) [2023-10-12 20:53:03,901][44959] Updated weights for policy 1, policy_version 22040 (0.0007) [2023-10-12 20:53:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45023232. Throughput: 0: 1641.3, 1: 1648.6. Samples: 11262878. Policy #0 lag: (min: 6.0, avg: 15.3, max: 38.0) [2023-10-12 20:53:06,443][43579] Avg episode reward: [(0, '271.080'), (1, '281.360')] [2023-10-12 20:53:06,728][44958] Updated weights for policy 0, policy_version 21930 (0.0008) [2023-10-12 20:53:07,105][44958] Updated weights for policy 0, policy_version 21940 (0.0008) [2023-10-12 20:53:07,482][44958] Updated weights for policy 0, policy_version 21950 (0.0009) [2023-10-12 20:53:07,916][44959] Updated weights for policy 1, policy_version 22050 (0.0008) [2023-10-12 20:53:08,281][44959] Updated weights for policy 1, policy_version 22060 (0.0008) [2023-10-12 20:53:08,647][44959] Updated weights for policy 1, policy_version 22070 (0.0009) [2023-10-12 20:53:09,014][44959] Updated weights for policy 1, policy_version 22080 (0.0008) [2023-10-12 20:53:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45088768. Throughput: 0: 1633.3, 1: 1664.1. Samples: 11283036. Policy #0 lag: (min: 6.0, avg: 15.3, max: 38.0) [2023-10-12 20:53:11,443][43579] Avg episode reward: [(0, '271.250'), (1, '282.550')] [2023-10-12 20:53:11,757][44958] Updated weights for policy 0, policy_version 21960 (0.0008) [2023-10-12 20:53:12,121][44958] Updated weights for policy 0, policy_version 21970 (0.0009) [2023-10-12 20:53:12,487][44958] Updated weights for policy 0, policy_version 21980 (0.0009) [2023-10-12 20:53:13,099][44959] Updated weights for policy 1, policy_version 22090 (0.0008) [2023-10-12 20:53:13,472][44959] Updated weights for policy 1, policy_version 22100 (0.0010) [2023-10-12 20:53:13,835][44959] Updated weights for policy 1, policy_version 22110 (0.0008) [2023-10-12 20:53:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45154304. Throughput: 0: 1635.0, 1: 1672.1. Samples: 11303498. Policy #0 lag: (min: 6.0, avg: 15.3, max: 38.0) [2023-10-12 20:53:16,443][43579] Avg episode reward: [(0, '276.560'), (1, '275.550')] [2023-10-12 20:53:16,779][44958] Updated weights for policy 0, policy_version 21990 (0.0008) [2023-10-12 20:53:17,162][44958] Updated weights for policy 0, policy_version 22000 (0.0009) [2023-10-12 20:53:17,535][44958] Updated weights for policy 0, policy_version 22010 (0.0009) [2023-10-12 20:53:17,968][44959] Updated weights for policy 1, policy_version 22120 (0.0008) [2023-10-12 20:53:18,340][44959] Updated weights for policy 1, policy_version 22130 (0.0008) [2023-10-12 20:53:18,717][44959] Updated weights for policy 1, policy_version 22140 (0.0008) [2023-10-12 20:53:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45219840. Throughput: 0: 1632.4, 1: 1654.6. Samples: 11312110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-12 20:53:21,443][43579] Avg episode reward: [(0, '274.990'), (1, '274.520')] [2023-10-12 20:53:21,824][44958] Updated weights for policy 0, policy_version 22020 (0.0009) [2023-10-12 20:53:22,214][44958] Updated weights for policy 0, policy_version 22030 (0.0009) [2023-10-12 20:53:22,588][44958] Updated weights for policy 0, policy_version 22040 (0.0008) [2023-10-12 20:53:22,907][44959] Updated weights for policy 1, policy_version 22150 (0.0008) [2023-10-12 20:53:23,276][44959] Updated weights for policy 1, policy_version 22160 (0.0009) [2023-10-12 20:53:23,644][44959] Updated weights for policy 1, policy_version 22170 (0.0009) [2023-10-12 20:53:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45285376. Throughput: 0: 1625.6, 1: 1664.8. Samples: 11332044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-12 20:53:26,443][43579] Avg episode reward: [(0, '276.180'), (1, '272.100')] [2023-10-12 20:53:26,792][44958] Updated weights for policy 0, policy_version 22050 (0.0008) [2023-10-12 20:53:27,165][44958] Updated weights for policy 0, policy_version 22060 (0.0010) [2023-10-12 20:53:27,545][44958] Updated weights for policy 0, policy_version 22070 (0.0007) [2023-10-12 20:53:27,924][44958] Updated weights for policy 0, policy_version 22080 (0.0007) [2023-10-12 20:53:28,013][44959] Updated weights for policy 1, policy_version 22180 (0.0008) [2023-10-12 20:53:28,419][44959] Updated weights for policy 1, policy_version 22190 (0.0010) [2023-10-12 20:53:28,792][44959] Updated weights for policy 1, policy_version 22200 (0.0010) [2023-10-12 20:53:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45350912. Throughput: 0: 1628.4, 1: 1657.3. Samples: 11351998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-12 20:53:31,443][43579] Avg episode reward: [(0, '275.420'), (1, '269.070')] [2023-10-12 20:53:32,144][44958] Updated weights for policy 0, policy_version 22090 (0.0010) [2023-10-12 20:53:32,521][44958] Updated weights for policy 0, policy_version 22100 (0.0007) [2023-10-12 20:53:32,848][44959] Updated weights for policy 1, policy_version 22210 (0.0010) [2023-10-12 20:53:32,901][44958] Updated weights for policy 0, policy_version 22110 (0.0009) [2023-10-12 20:53:33,221][44959] Updated weights for policy 1, policy_version 22220 (0.0008) [2023-10-12 20:53:33,589][44959] Updated weights for policy 1, policy_version 22230 (0.0010) [2023-10-12 20:53:33,950][44959] Updated weights for policy 1, policy_version 22240 (0.0009) [2023-10-12 20:53:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45416448. Throughput: 0: 1630.9, 1: 1647.0. Samples: 11360918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-12 20:53:36,443][43579] Avg episode reward: [(0, '274.960'), (1, '267.590')] [2023-10-12 20:53:36,940][44958] Updated weights for policy 0, policy_version 22120 (0.0009) [2023-10-12 20:53:37,318][44958] Updated weights for policy 0, policy_version 22130 (0.0008) [2023-10-12 20:53:37,694][44958] Updated weights for policy 0, policy_version 22140 (0.0010) [2023-10-12 20:53:38,068][44959] Updated weights for policy 1, policy_version 22250 (0.0009) [2023-10-12 20:53:38,431][44959] Updated weights for policy 1, policy_version 22260 (0.0008) [2023-10-12 20:53:38,799][44959] Updated weights for policy 1, policy_version 22270 (0.0007) [2023-10-12 20:53:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45481984. Throughput: 0: 1625.8, 1: 1664.4. Samples: 11381248. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-10-12 20:53:41,444][43579] Avg episode reward: [(0, '276.610'), (1, '267.640')] [2023-10-12 20:53:41,837][44958] Updated weights for policy 0, policy_version 22150 (0.0010) [2023-10-12 20:53:42,209][44958] Updated weights for policy 0, policy_version 22160 (0.0008) [2023-10-12 20:53:42,584][44958] Updated weights for policy 0, policy_version 22170 (0.0008) [2023-10-12 20:53:42,892][44959] Updated weights for policy 1, policy_version 22280 (0.0008) [2023-10-12 20:53:43,264][44959] Updated weights for policy 1, policy_version 22290 (0.0007) [2023-10-12 20:53:43,627][44959] Updated weights for policy 1, policy_version 22300 (0.0008) [2023-10-12 20:53:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 45547520. Throughput: 0: 1631.2, 1: 1658.5. Samples: 11401644. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-10-12 20:53:46,444][43579] Avg episode reward: [(0, '272.730'), (1, '272.280')] [2023-10-12 20:53:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000022176_22708224.pth... [2023-10-12 20:53:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000022304_22839296.pth... [2023-10-12 20:53:46,489][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000020640_21135360.pth [2023-10-12 20:53:46,491][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000020768_21266432.pth [2023-10-12 20:53:46,772][44958] Updated weights for policy 0, policy_version 22180 (0.0009) [2023-10-12 20:53:47,159][44958] Updated weights for policy 0, policy_version 22190 (0.0009) [2023-10-12 20:53:47,526][44958] Updated weights for policy 0, policy_version 22200 (0.0008) [2023-10-12 20:53:47,928][44959] Updated weights for policy 1, policy_version 22310 (0.0008) [2023-10-12 20:53:48,288][44959] Updated weights for policy 1, policy_version 22320 (0.0009) [2023-10-12 20:53:48,659][44959] Updated weights for policy 1, policy_version 22330 (0.0007) [2023-10-12 20:53:51,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45613056. Throughput: 0: 1630.8, 1: 1648.1. Samples: 11410432. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-10-12 20:53:51,443][43579] Avg episode reward: [(0, '263.480'), (1, '274.330')] [2023-10-12 20:53:51,662][44958] Updated weights for policy 0, policy_version 22210 (0.0008) [2023-10-12 20:53:52,038][44958] Updated weights for policy 0, policy_version 22220 (0.0008) [2023-10-12 20:53:52,409][44958] Updated weights for policy 0, policy_version 22230 (0.0007) [2023-10-12 20:53:52,669][44959] Updated weights for policy 1, policy_version 22340 (0.0007) [2023-10-12 20:53:52,777][44958] Updated weights for policy 0, policy_version 22240 (0.0007) [2023-10-12 20:53:53,032][44959] Updated weights for policy 1, policy_version 22350 (0.0007) [2023-10-12 20:53:53,396][44959] Updated weights for policy 1, policy_version 22360 (0.0008) [2023-10-12 20:53:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45678592. Throughput: 0: 1634.6, 1: 1647.4. Samples: 11430726. Policy #0 lag: (min: 2.0, avg: 12.3, max: 34.0) [2023-10-12 20:53:56,443][43579] Avg episode reward: [(0, '262.620'), (1, '276.770')] [2023-10-12 20:53:56,868][44958] Updated weights for policy 0, policy_version 22250 (0.0010) [2023-10-12 20:53:57,226][44958] Updated weights for policy 0, policy_version 22260 (0.0010) [2023-10-12 20:53:57,607][44958] Updated weights for policy 0, policy_version 22270 (0.0008) [2023-10-12 20:53:57,752][44959] Updated weights for policy 1, policy_version 22370 (0.0009) [2023-10-12 20:53:58,118][44959] Updated weights for policy 1, policy_version 22380 (0.0009) [2023-10-12 20:53:58,480][44959] Updated weights for policy 1, policy_version 22390 (0.0010) [2023-10-12 20:53:58,844][44959] Updated weights for policy 1, policy_version 22400 (0.0011) [2023-10-12 20:54:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45744128. Throughput: 0: 1633.7, 1: 1642.2. Samples: 11450916. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-12 20:54:01,443][43579] Avg episode reward: [(0, '260.310'), (1, '280.260')] [2023-10-12 20:54:01,766][44958] Updated weights for policy 0, policy_version 22280 (0.0008) [2023-10-12 20:54:02,148][44958] Updated weights for policy 0, policy_version 22290 (0.0007) [2023-10-12 20:54:02,521][44958] Updated weights for policy 0, policy_version 22300 (0.0008) [2023-10-12 20:54:02,808][44959] Updated weights for policy 1, policy_version 22410 (0.0009) [2023-10-12 20:54:03,189][44959] Updated weights for policy 1, policy_version 22420 (0.0007) [2023-10-12 20:54:03,566][44959] Updated weights for policy 1, policy_version 22430 (0.0010) [2023-10-12 20:54:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45809664. Throughput: 0: 1633.3, 1: 1648.1. Samples: 11459776. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-12 20:54:06,443][43579] Avg episode reward: [(0, '259.990'), (1, '280.210')] [2023-10-12 20:54:06,851][44958] Updated weights for policy 0, policy_version 22310 (0.0009) [2023-10-12 20:54:07,222][44958] Updated weights for policy 0, policy_version 22320 (0.0009) [2023-10-12 20:54:07,593][44958] Updated weights for policy 0, policy_version 22330 (0.0009) [2023-10-12 20:54:08,003][44959] Updated weights for policy 1, policy_version 22440 (0.0009) [2023-10-12 20:54:08,373][44959] Updated weights for policy 1, policy_version 22450 (0.0009) [2023-10-12 20:54:08,727][44959] Updated weights for policy 1, policy_version 22460 (0.0007) [2023-10-12 20:54:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45875200. Throughput: 0: 1638.8, 1: 1643.3. Samples: 11479740. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-12 20:54:11,444][43579] Avg episode reward: [(0, '264.490'), (1, '283.510')] [2023-10-12 20:54:11,638][44958] Updated weights for policy 0, policy_version 22340 (0.0009) [2023-10-12 20:54:12,014][44958] Updated weights for policy 0, policy_version 22350 (0.0008) [2023-10-12 20:54:12,383][44958] Updated weights for policy 0, policy_version 22360 (0.0008) [2023-10-12 20:54:12,971][44959] Updated weights for policy 1, policy_version 22470 (0.0007) [2023-10-12 20:54:13,352][44959] Updated weights for policy 1, policy_version 22480 (0.0007) [2023-10-12 20:54:13,720][44959] Updated weights for policy 1, policy_version 22490 (0.0008) [2023-10-12 20:54:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 45940736. Throughput: 0: 1639.4, 1: 1643.3. Samples: 11499720. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-12 20:54:16,444][43579] Avg episode reward: [(0, '266.090'), (1, '284.980')] [2023-10-12 20:54:16,528][44958] Updated weights for policy 0, policy_version 22370 (0.0008) [2023-10-12 20:54:16,904][44958] Updated weights for policy 0, policy_version 22380 (0.0007) [2023-10-12 20:54:17,273][44958] Updated weights for policy 0, policy_version 22390 (0.0008) [2023-10-12 20:54:17,651][44958] Updated weights for policy 0, policy_version 22400 (0.0009) [2023-10-12 20:54:17,694][44959] Updated weights for policy 1, policy_version 22500 (0.0008) [2023-10-12 20:54:18,060][44959] Updated weights for policy 1, policy_version 22510 (0.0007) [2023-10-12 20:54:18,435][44959] Updated weights for policy 1, policy_version 22520 (0.0009) [2023-10-12 20:54:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46006272. Throughput: 0: 1637.5, 1: 1645.2. Samples: 11508640. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-12 20:54:21,443][43579] Avg episode reward: [(0, '271.160'), (1, '280.560')] [2023-10-12 20:54:21,824][44958] Updated weights for policy 0, policy_version 22410 (0.0009) [2023-10-12 20:54:22,195][44958] Updated weights for policy 0, policy_version 22420 (0.0008) [2023-10-12 20:54:22,576][44958] Updated weights for policy 0, policy_version 22430 (0.0008) [2023-10-12 20:54:22,887][44959] Updated weights for policy 1, policy_version 22530 (0.0008) [2023-10-12 20:54:23,268][44959] Updated weights for policy 1, policy_version 22540 (0.0008) [2023-10-12 20:54:23,632][44959] Updated weights for policy 1, policy_version 22550 (0.0008) [2023-10-12 20:54:23,989][44959] Updated weights for policy 1, policy_version 22560 (0.0007) [2023-10-12 20:54:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46071808. Throughput: 0: 1639.7, 1: 1639.3. Samples: 11528802. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-12 20:54:26,444][43579] Avg episode reward: [(0, '269.820'), (1, '283.740')] [2023-10-12 20:54:26,703][44958] Updated weights for policy 0, policy_version 22440 (0.0008) [2023-10-12 20:54:27,069][44958] Updated weights for policy 0, policy_version 22450 (0.0010) [2023-10-12 20:54:27,454][44958] Updated weights for policy 0, policy_version 22460 (0.0010) [2023-10-12 20:54:28,117][44959] Updated weights for policy 1, policy_version 22570 (0.0008) [2023-10-12 20:54:28,485][44959] Updated weights for policy 1, policy_version 22580 (0.0009) [2023-10-12 20:54:28,860][44959] Updated weights for policy 1, policy_version 22590 (0.0009) [2023-10-12 20:54:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46137344. Throughput: 0: 1635.6, 1: 1645.7. Samples: 11549300. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-12 20:54:31,443][43579] Avg episode reward: [(0, '268.840'), (1, '282.750')] [2023-10-12 20:54:31,467][44958] Updated weights for policy 0, policy_version 22470 (0.0009) [2023-10-12 20:54:31,842][44958] Updated weights for policy 0, policy_version 22480 (0.0007) [2023-10-12 20:54:32,205][44958] Updated weights for policy 0, policy_version 22490 (0.0011) [2023-10-12 20:54:32,766][44959] Updated weights for policy 1, policy_version 22600 (0.0009) [2023-10-12 20:54:33,131][44959] Updated weights for policy 1, policy_version 22610 (0.0009) [2023-10-12 20:54:33,499][44959] Updated weights for policy 1, policy_version 22620 (0.0010) [2023-10-12 20:54:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46202880. Throughput: 0: 1637.0, 1: 1648.0. Samples: 11558260. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) [2023-10-12 20:54:36,444][43579] Avg episode reward: [(0, '267.090'), (1, '281.710')] [2023-10-12 20:54:36,617][44958] Updated weights for policy 0, policy_version 22500 (0.0010) [2023-10-12 20:54:36,994][44958] Updated weights for policy 0, policy_version 22510 (0.0009) [2023-10-12 20:54:37,365][44958] Updated weights for policy 0, policy_version 22520 (0.0009) [2023-10-12 20:54:37,620][44959] Updated weights for policy 1, policy_version 22630 (0.0009) [2023-10-12 20:54:37,991][44959] Updated weights for policy 1, policy_version 22640 (0.0008) [2023-10-12 20:54:38,355][44959] Updated weights for policy 1, policy_version 22650 (0.0007) [2023-10-12 20:54:41,376][44958] Updated weights for policy 0, policy_version 22530 (0.0009) [2023-10-12 20:54:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46268416. Throughput: 0: 1634.6, 1: 1649.8. Samples: 11578522. Policy #0 lag: (min: 9.0, avg: 21.1, max: 41.0) [2023-10-12 20:54:41,444][43579] Avg episode reward: [(0, '260.740'), (1, '278.800')] [2023-10-12 20:54:41,753][44958] Updated weights for policy 0, policy_version 22540 (0.0010) [2023-10-12 20:54:42,125][44958] Updated weights for policy 0, policy_version 22550 (0.0009) [2023-10-12 20:54:42,496][44958] Updated weights for policy 0, policy_version 22560 (0.0010) [2023-10-12 20:54:42,549][44959] Updated weights for policy 1, policy_version 22660 (0.0009) [2023-10-12 20:54:42,918][44959] Updated weights for policy 1, policy_version 22670 (0.0011) [2023-10-12 20:54:43,283][44959] Updated weights for policy 1, policy_version 22680 (0.0008) [2023-10-12 20:54:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46333952. Throughput: 0: 1637.8, 1: 1649.6. Samples: 11598852. Policy #0 lag: (min: 9.0, avg: 21.1, max: 41.0) [2023-10-12 20:54:46,444][43579] Avg episode reward: [(0, '261.210'), (1, '276.360')] [2023-10-12 20:54:46,625][44958] Updated weights for policy 0, policy_version 22570 (0.0011) [2023-10-12 20:54:47,014][44958] Updated weights for policy 0, policy_version 22580 (0.0011) [2023-10-12 20:54:47,386][44958] Updated weights for policy 0, policy_version 22590 (0.0010) [2023-10-12 20:54:47,398][44959] Updated weights for policy 1, policy_version 22690 (0.0009) [2023-10-12 20:54:47,764][44959] Updated weights for policy 1, policy_version 22700 (0.0007) [2023-10-12 20:54:48,127][44959] Updated weights for policy 1, policy_version 22710 (0.0007) [2023-10-12 20:54:48,495][44959] Updated weights for policy 1, policy_version 22720 (0.0010) [2023-10-12 20:54:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46399488. Throughput: 0: 1634.2, 1: 1646.4. Samples: 11607404. Policy #0 lag: (min: 9.0, avg: 21.1, max: 41.0) [2023-10-12 20:54:51,443][43579] Avg episode reward: [(0, '264.070'), (1, '279.340')] [2023-10-12 20:54:51,796][44958] Updated weights for policy 0, policy_version 22600 (0.0010) [2023-10-12 20:54:52,169][44958] Updated weights for policy 0, policy_version 22610 (0.0010) [2023-10-12 20:54:52,548][44958] Updated weights for policy 0, policy_version 22620 (0.0008) [2023-10-12 20:54:52,629][44959] Updated weights for policy 1, policy_version 22730 (0.0007) [2023-10-12 20:54:52,997][44959] Updated weights for policy 1, policy_version 22740 (0.0007) [2023-10-12 20:54:53,359][44959] Updated weights for policy 1, policy_version 22750 (0.0007) [2023-10-12 20:54:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46465024. Throughput: 0: 1631.5, 1: 1652.5. Samples: 11627518. Policy #0 lag: (min: 9.0, avg: 21.1, max: 41.0) [2023-10-12 20:54:56,443][43579] Avg episode reward: [(0, '263.240'), (1, '277.460')] [2023-10-12 20:54:56,941][44958] Updated weights for policy 0, policy_version 22630 (0.0007) [2023-10-12 20:54:57,317][44958] Updated weights for policy 0, policy_version 22640 (0.0008) [2023-10-12 20:54:57,552][44959] Updated weights for policy 1, policy_version 22760 (0.0009) [2023-10-12 20:54:57,685][44958] Updated weights for policy 0, policy_version 22650 (0.0009) [2023-10-12 20:54:57,924][44959] Updated weights for policy 1, policy_version 22770 (0.0009) [2023-10-12 20:54:58,290][44959] Updated weights for policy 1, policy_version 22780 (0.0008) [2023-10-12 20:55:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46530560. Throughput: 0: 1640.5, 1: 1663.1. Samples: 11648384. Policy #0 lag: (min: 7.0, avg: 10.3, max: 39.0) [2023-10-12 20:55:01,444][43579] Avg episode reward: [(0, '266.320'), (1, '274.770')] [2023-10-12 20:55:01,791][44958] Updated weights for policy 0, policy_version 22660 (0.0008) [2023-10-12 20:55:02,168][44958] Updated weights for policy 0, policy_version 22670 (0.0009) [2023-10-12 20:55:02,319][44959] Updated weights for policy 1, policy_version 22790 (0.0008) [2023-10-12 20:55:02,531][44958] Updated weights for policy 0, policy_version 22680 (0.0007) [2023-10-12 20:55:02,693][44959] Updated weights for policy 1, policy_version 22800 (0.0007) [2023-10-12 20:55:03,059][44959] Updated weights for policy 1, policy_version 22810 (0.0008) [2023-10-12 20:55:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46596096. Throughput: 0: 1639.1, 1: 1659.8. Samples: 11657092. Policy #0 lag: (min: 7.0, avg: 10.3, max: 39.0) [2023-10-12 20:55:06,443][43579] Avg episode reward: [(0, '261.470'), (1, '277.240')] [2023-10-12 20:55:06,497][44958] Updated weights for policy 0, policy_version 22690 (0.0007) [2023-10-12 20:55:06,868][44958] Updated weights for policy 0, policy_version 22700 (0.0008) [2023-10-12 20:55:07,166][44959] Updated weights for policy 1, policy_version 22820 (0.0010) [2023-10-12 20:55:07,232][44958] Updated weights for policy 0, policy_version 22710 (0.0008) [2023-10-12 20:55:07,535][44959] Updated weights for policy 1, policy_version 22830 (0.0009) [2023-10-12 20:55:07,603][44958] Updated weights for policy 0, policy_version 22720 (0.0009) [2023-10-12 20:55:07,896][44959] Updated weights for policy 1, policy_version 22840 (0.0009) [2023-10-12 20:55:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46661632. Throughput: 0: 1643.1, 1: 1660.2. Samples: 11677452. Policy #0 lag: (min: 7.0, avg: 10.3, max: 39.0) [2023-10-12 20:55:11,443][43579] Avg episode reward: [(0, '264.570'), (1, '280.160')] [2023-10-12 20:55:11,794][44958] Updated weights for policy 0, policy_version 22730 (0.0009) [2023-10-12 20:55:11,975][44959] Updated weights for policy 1, policy_version 22850 (0.0009) [2023-10-12 20:55:12,170][44958] Updated weights for policy 0, policy_version 22740 (0.0010) [2023-10-12 20:55:12,347][44959] Updated weights for policy 1, policy_version 22860 (0.0009) [2023-10-12 20:55:12,545][44958] Updated weights for policy 0, policy_version 22750 (0.0009) [2023-10-12 20:55:12,719][44959] Updated weights for policy 1, policy_version 22870 (0.0008) [2023-10-12 20:55:13,082][44959] Updated weights for policy 1, policy_version 22880 (0.0009) [2023-10-12 20:55:16,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46727168. Throughput: 0: 1648.6, 1: 1655.4. Samples: 11697982. Policy #0 lag: (min: 7.0, avg: 10.3, max: 39.0) [2023-10-12 20:55:16,443][43579] Avg episode reward: [(0, '261.790'), (1, '283.870')] [2023-10-12 20:55:16,471][44958] Updated weights for policy 0, policy_version 22760 (0.0010) [2023-10-12 20:55:16,851][44958] Updated weights for policy 0, policy_version 22770 (0.0009) [2023-10-12 20:55:17,221][44958] Updated weights for policy 0, policy_version 22780 (0.0009) [2023-10-12 20:55:17,348][44959] Updated weights for policy 1, policy_version 22890 (0.0007) [2023-10-12 20:55:17,717][44959] Updated weights for policy 1, policy_version 22900 (0.0008) [2023-10-12 20:55:18,091][44959] Updated weights for policy 1, policy_version 22910 (0.0009) [2023-10-12 20:55:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46792704. Throughput: 0: 1648.7, 1: 1651.9. Samples: 11706786. Policy #0 lag: (min: 14.0, avg: 19.2, max: 46.0) [2023-10-12 20:55:21,443][43579] Avg episode reward: [(0, '264.750'), (1, '280.730')] [2023-10-12 20:55:21,466][44958] Updated weights for policy 0, policy_version 22790 (0.0009) [2023-10-12 20:55:21,839][44958] Updated weights for policy 0, policy_version 22800 (0.0011) [2023-10-12 20:55:22,206][44958] Updated weights for policy 0, policy_version 22810 (0.0010) [2023-10-12 20:55:22,240][44959] Updated weights for policy 1, policy_version 22920 (0.0008) [2023-10-12 20:55:22,615][44959] Updated weights for policy 1, policy_version 22930 (0.0008) [2023-10-12 20:55:22,976][44959] Updated weights for policy 1, policy_version 22940 (0.0010) [2023-10-12 20:55:26,409][44958] Updated weights for policy 0, policy_version 22820 (0.0009) [2023-10-12 20:55:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46858240. Throughput: 0: 1646.8, 1: 1654.1. Samples: 11727066. Policy #0 lag: (min: 14.0, avg: 19.2, max: 46.0) [2023-10-12 20:55:26,444][43579] Avg episode reward: [(0, '267.480'), (1, '278.410')] [2023-10-12 20:55:26,789][44958] Updated weights for policy 0, policy_version 22830 (0.0009) [2023-10-12 20:55:27,056][44959] Updated weights for policy 1, policy_version 22950 (0.0008) [2023-10-12 20:55:27,165][44958] Updated weights for policy 0, policy_version 22840 (0.0009) [2023-10-12 20:55:27,425][44959] Updated weights for policy 1, policy_version 22960 (0.0007) [2023-10-12 20:55:27,795][44959] Updated weights for policy 1, policy_version 22970 (0.0007) [2023-10-12 20:55:31,420][44958] Updated weights for policy 0, policy_version 22850 (0.0009) [2023-10-12 20:55:31,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 46923776. Throughput: 0: 1638.9, 1: 1658.0. Samples: 11747214. Policy #0 lag: (min: 14.0, avg: 19.2, max: 46.0) [2023-10-12 20:55:31,444][43579] Avg episode reward: [(0, '263.660'), (1, '276.060')] [2023-10-12 20:55:31,792][44958] Updated weights for policy 0, policy_version 22860 (0.0008) [2023-10-12 20:55:31,988][44959] Updated weights for policy 1, policy_version 22980 (0.0009) [2023-10-12 20:55:32,164][44958] Updated weights for policy 0, policy_version 22870 (0.0009) [2023-10-12 20:55:32,372][44959] Updated weights for policy 1, policy_version 22990 (0.0009) [2023-10-12 20:55:32,530][44958] Updated weights for policy 0, policy_version 22880 (0.0008) [2023-10-12 20:55:32,741][44959] Updated weights for policy 1, policy_version 23000 (0.0008) [2023-10-12 20:55:36,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 46989312. Throughput: 0: 1643.6, 1: 1659.6. Samples: 11756044. Policy #0 lag: (min: 14.0, avg: 19.2, max: 46.0) [2023-10-12 20:55:36,443][43579] Avg episode reward: [(0, '266.170'), (1, '275.790')] [2023-10-12 20:55:36,841][44958] Updated weights for policy 0, policy_version 22890 (0.0007) [2023-10-12 20:55:36,847][44959] Updated weights for policy 1, policy_version 23010 (0.0008) [2023-10-12 20:55:37,216][44958] Updated weights for policy 0, policy_version 22900 (0.0008) [2023-10-12 20:55:37,219][44959] Updated weights for policy 1, policy_version 23020 (0.0008) [2023-10-12 20:55:37,579][44958] Updated weights for policy 0, policy_version 22910 (0.0008) [2023-10-12 20:55:37,594][44959] Updated weights for policy 1, policy_version 23030 (0.0009) [2023-10-12 20:55:37,960][44959] Updated weights for policy 1, policy_version 23040 (0.0008) [2023-10-12 20:55:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47054848. Throughput: 0: 1641.5, 1: 1662.7. Samples: 11776206. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-12 20:55:41,444][43579] Avg episode reward: [(0, '267.780'), (1, '272.430')] [2023-10-12 20:55:41,676][44958] Updated weights for policy 0, policy_version 22920 (0.0007) [2023-10-12 20:55:42,002][44959] Updated weights for policy 1, policy_version 23050 (0.0008) [2023-10-12 20:55:42,058][44958] Updated weights for policy 0, policy_version 22930 (0.0009) [2023-10-12 20:55:42,368][44959] Updated weights for policy 1, policy_version 23060 (0.0008) [2023-10-12 20:55:42,423][44958] Updated weights for policy 0, policy_version 22940 (0.0007) [2023-10-12 20:55:42,727][44959] Updated weights for policy 1, policy_version 23070 (0.0009) [2023-10-12 20:55:46,443][43579] Fps is (10 sec: 13106.4, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 47120384. Throughput: 0: 1630.3, 1: 1662.9. Samples: 11796578. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-12 20:55:46,445][43579] Avg episode reward: [(0, '272.630'), (1, '268.430')] [2023-10-12 20:55:46,457][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000022944_23494656.pth... [2023-10-12 20:55:46,493][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000021440_21954560.pth [2023-10-12 20:55:46,497][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000022944_23494656.pth [2023-10-12 20:55:46,867][44959] Updated weights for policy 1, policy_version 23080 (0.0008) [2023-10-12 20:55:46,881][44958] Updated weights for policy 0, policy_version 22950 (0.0008) [2023-10-12 20:55:47,245][44959] Updated weights for policy 1, policy_version 23090 (0.0008) [2023-10-12 20:55:47,247][44958] Updated weights for policy 0, policy_version 22960 (0.0008) [2023-10-12 20:55:47,617][44959] Updated weights for policy 1, policy_version 23100 (0.0008) [2023-10-12 20:55:47,622][44958] Updated weights for policy 0, policy_version 22970 (0.0008) [2023-10-12 20:55:47,761][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000023104_23658496.pth... [2023-10-12 20:55:47,790][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000021536_22052864.pth [2023-10-12 20:55:47,794][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000023104_23658496.pth [2023-10-12 20:55:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47185920. Throughput: 0: 1627.5, 1: 1659.6. Samples: 11805014. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-12 20:55:51,444][43579] Avg episode reward: [(0, '271.740'), (1, '261.260')] [2023-10-12 20:55:51,848][44959] Updated weights for policy 1, policy_version 23110 (0.0008) [2023-10-12 20:55:51,870][44958] Updated weights for policy 0, policy_version 22980 (0.0008) [2023-10-12 20:55:52,212][44959] Updated weights for policy 1, policy_version 23120 (0.0010) [2023-10-12 20:55:52,236][44958] Updated weights for policy 0, policy_version 22990 (0.0008) [2023-10-12 20:55:52,585][44959] Updated weights for policy 1, policy_version 23130 (0.0009) [2023-10-12 20:55:52,613][44958] Updated weights for policy 0, policy_version 23000 (0.0007) [2023-10-12 20:55:56,443][43579] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47251456. Throughput: 0: 1621.2, 1: 1664.0. Samples: 11825286. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-12 20:55:56,444][43579] Avg episode reward: [(0, '266.810'), (1, '267.350')] [2023-10-12 20:55:56,587][44959] Updated weights for policy 1, policy_version 23140 (0.0010) [2023-10-12 20:55:56,753][44958] Updated weights for policy 0, policy_version 23010 (0.0007) [2023-10-12 20:55:56,964][44959] Updated weights for policy 1, policy_version 23150 (0.0009) [2023-10-12 20:55:57,125][44958] Updated weights for policy 0, policy_version 23020 (0.0008) [2023-10-12 20:55:57,332][44959] Updated weights for policy 1, policy_version 23160 (0.0008) [2023-10-12 20:55:57,496][44958] Updated weights for policy 0, policy_version 23030 (0.0009) [2023-10-12 20:55:57,863][44958] Updated weights for policy 0, policy_version 23040 (0.0010) [2023-10-12 20:56:01,442][44959] Updated weights for policy 1, policy_version 23170 (0.0007) [2023-10-12 20:56:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47316992. Throughput: 0: 1612.1, 1: 1664.2. Samples: 11845414. Policy #0 lag: (min: 2.0, avg: 4.7, max: 34.0) [2023-10-12 20:56:01,443][43579] Avg episode reward: [(0, '267.290'), (1, '269.240')] [2023-10-12 20:56:01,810][44959] Updated weights for policy 1, policy_version 23180 (0.0009) [2023-10-12 20:56:02,023][44958] Updated weights for policy 0, policy_version 23050 (0.0009) [2023-10-12 20:56:02,175][44959] Updated weights for policy 1, policy_version 23190 (0.0008) [2023-10-12 20:56:02,389][44958] Updated weights for policy 0, policy_version 23060 (0.0007) [2023-10-12 20:56:02,541][44959] Updated weights for policy 1, policy_version 23200 (0.0009) [2023-10-12 20:56:02,769][44958] Updated weights for policy 0, policy_version 23070 (0.0010) [2023-10-12 20:56:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47382528. Throughput: 0: 1611.6, 1: 1663.3. Samples: 11854154. Policy #0 lag: (min: 2.0, avg: 4.7, max: 34.0) [2023-10-12 20:56:06,443][43579] Avg episode reward: [(0, '262.230'), (1, '270.900')] [2023-10-12 20:56:06,725][44959] Updated weights for policy 1, policy_version 23210 (0.0008) [2023-10-12 20:56:07,090][44959] Updated weights for policy 1, policy_version 23220 (0.0008) [2023-10-12 20:56:07,140][44958] Updated weights for policy 0, policy_version 23080 (0.0009) [2023-10-12 20:56:07,462][44959] Updated weights for policy 1, policy_version 23230 (0.0007) [2023-10-12 20:56:07,512][44958] Updated weights for policy 0, policy_version 23090 (0.0007) [2023-10-12 20:56:07,889][44958] Updated weights for policy 0, policy_version 23100 (0.0010) [2023-10-12 20:56:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47448064. Throughput: 0: 1609.8, 1: 1659.3. Samples: 11874178. Policy #0 lag: (min: 2.0, avg: 4.7, max: 34.0) [2023-10-12 20:56:11,444][43579] Avg episode reward: [(0, '262.160'), (1, '269.350')] [2023-10-12 20:56:11,865][44959] Updated weights for policy 1, policy_version 23240 (0.0009) [2023-10-12 20:56:12,226][44959] Updated weights for policy 1, policy_version 23250 (0.0009) [2023-10-12 20:56:12,268][44958] Updated weights for policy 0, policy_version 23110 (0.0010) [2023-10-12 20:56:12,602][44959] Updated weights for policy 1, policy_version 23260 (0.0008) [2023-10-12 20:56:12,636][44958] Updated weights for policy 0, policy_version 23120 (0.0009) [2023-10-12 20:56:13,006][44958] Updated weights for policy 0, policy_version 23130 (0.0008) [2023-10-12 20:56:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47513600. Throughput: 0: 1619.8, 1: 1655.0. Samples: 11894582. Policy #0 lag: (min: 2.0, avg: 4.7, max: 34.0) [2023-10-12 20:56:16,443][43579] Avg episode reward: [(0, '263.880'), (1, '267.930')] [2023-10-12 20:56:16,854][44959] Updated weights for policy 1, policy_version 23270 (0.0011) [2023-10-12 20:56:17,219][44959] Updated weights for policy 1, policy_version 23280 (0.0010) [2023-10-12 20:56:17,326][44958] Updated weights for policy 0, policy_version 23140 (0.0008) [2023-10-12 20:56:17,587][44959] Updated weights for policy 1, policy_version 23290 (0.0008) [2023-10-12 20:56:17,722][44958] Updated weights for policy 0, policy_version 23150 (0.0009) [2023-10-12 20:56:18,093][44958] Updated weights for policy 0, policy_version 23160 (0.0008) [2023-10-12 20:56:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 47579136. Throughput: 0: 1618.9, 1: 1653.2. Samples: 11903290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:56:21,444][43579] Avg episode reward: [(0, '259.390'), (1, '276.320')] [2023-10-12 20:56:21,712][44959] Updated weights for policy 1, policy_version 23300 (0.0008) [2023-10-12 20:56:22,083][44959] Updated weights for policy 1, policy_version 23310 (0.0007) [2023-10-12 20:56:22,127][44958] Updated weights for policy 0, policy_version 23170 (0.0010) [2023-10-12 20:56:22,456][44959] Updated weights for policy 1, policy_version 23320 (0.0008) [2023-10-12 20:56:22,499][44958] Updated weights for policy 0, policy_version 23180 (0.0008) [2023-10-12 20:56:22,877][44958] Updated weights for policy 0, policy_version 23190 (0.0010) [2023-10-12 20:56:23,250][44958] Updated weights for policy 0, policy_version 23200 (0.0007) [2023-10-12 20:56:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47644672. Throughput: 0: 1619.9, 1: 1645.7. Samples: 11923158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:56:26,444][43579] Avg episode reward: [(0, '261.160'), (1, '274.020')] [2023-10-12 20:56:26,659][44959] Updated weights for policy 1, policy_version 23330 (0.0009) [2023-10-12 20:56:27,026][44959] Updated weights for policy 1, policy_version 23340 (0.0009) [2023-10-12 20:56:27,399][44959] Updated weights for policy 1, policy_version 23350 (0.0010) [2023-10-12 20:56:27,581][44958] Updated weights for policy 0, policy_version 23210 (0.0010) [2023-10-12 20:56:27,763][44959] Updated weights for policy 1, policy_version 23360 (0.0007) [2023-10-12 20:56:27,949][44958] Updated weights for policy 0, policy_version 23220 (0.0009) [2023-10-12 20:56:28,330][44958] Updated weights for policy 0, policy_version 23230 (0.0007) [2023-10-12 20:56:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 47710208. Throughput: 0: 1623.8, 1: 1639.1. Samples: 11943408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:56:31,443][43579] Avg episode reward: [(0, '265.070'), (1, '274.380')] [2023-10-12 20:56:32,174][44959] Updated weights for policy 1, policy_version 23370 (0.0010) [2023-10-12 20:56:32,278][44958] Updated weights for policy 0, policy_version 23240 (0.0010) [2023-10-12 20:56:32,551][44959] Updated weights for policy 1, policy_version 23380 (0.0008) [2023-10-12 20:56:32,655][44958] Updated weights for policy 0, policy_version 23250 (0.0008) [2023-10-12 20:56:32,920][44959] Updated weights for policy 1, policy_version 23390 (0.0010) [2023-10-12 20:56:33,026][44958] Updated weights for policy 0, policy_version 23260 (0.0007) [2023-10-12 20:56:36,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47775744. Throughput: 0: 1626.4, 1: 1641.2. Samples: 11952056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 20:56:36,443][43579] Avg episode reward: [(0, '267.570'), (1, '273.790')] [2023-10-12 20:56:36,905][44959] Updated weights for policy 1, policy_version 23400 (0.0008) [2023-10-12 20:56:37,140][44958] Updated weights for policy 0, policy_version 23270 (0.0009) [2023-10-12 20:56:37,269][44959] Updated weights for policy 1, policy_version 23410 (0.0009) [2023-10-12 20:56:37,509][44958] Updated weights for policy 0, policy_version 23280 (0.0009) [2023-10-12 20:56:37,630][44959] Updated weights for policy 1, policy_version 23420 (0.0011) [2023-10-12 20:56:37,881][44958] Updated weights for policy 0, policy_version 23290 (0.0008) [2023-10-12 20:56:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47841280. Throughput: 0: 1630.5, 1: 1636.2. Samples: 11972286. Policy #0 lag: (min: 27.0, avg: 34.8, max: 59.0) [2023-10-12 20:56:41,444][43579] Avg episode reward: [(0, '264.510'), (1, '279.600')] [2023-10-12 20:56:41,661][44959] Updated weights for policy 1, policy_version 23430 (0.0009) [2023-10-12 20:56:42,020][44959] Updated weights for policy 1, policy_version 23440 (0.0008) [2023-10-12 20:56:42,175][44958] Updated weights for policy 0, policy_version 23300 (0.0009) [2023-10-12 20:56:42,386][44959] Updated weights for policy 1, policy_version 23450 (0.0010) [2023-10-12 20:56:42,544][44958] Updated weights for policy 0, policy_version 23310 (0.0008) [2023-10-12 20:56:42,920][44958] Updated weights for policy 0, policy_version 23320 (0.0007) [2023-10-12 20:56:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 47906816. Throughput: 0: 1637.3, 1: 1635.9. Samples: 11992708. Policy #0 lag: (min: 27.0, avg: 34.8, max: 59.0) [2023-10-12 20:56:46,443][43579] Avg episode reward: [(0, '265.270'), (1, '282.300')] [2023-10-12 20:56:46,615][44959] Updated weights for policy 1, policy_version 23460 (0.0008) [2023-10-12 20:56:46,911][44958] Updated weights for policy 0, policy_version 23330 (0.0010) [2023-10-12 20:56:46,983][44959] Updated weights for policy 1, policy_version 23470 (0.0008) [2023-10-12 20:56:47,284][44958] Updated weights for policy 0, policy_version 23340 (0.0008) [2023-10-12 20:56:47,345][44959] Updated weights for policy 1, policy_version 23480 (0.0008) [2023-10-12 20:56:47,657][44958] Updated weights for policy 0, policy_version 23350 (0.0008) [2023-10-12 20:56:48,016][44958] Updated weights for policy 0, policy_version 23360 (0.0010) [2023-10-12 20:56:51,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 47972352. Throughput: 0: 1635.2, 1: 1641.6. Samples: 12001610. Policy #0 lag: (min: 27.0, avg: 34.8, max: 59.0) [2023-10-12 20:56:51,443][43579] Avg episode reward: [(0, '268.180'), (1, '279.330')] [2023-10-12 20:56:51,486][44959] Updated weights for policy 1, policy_version 23490 (0.0008) [2023-10-12 20:56:51,857][44959] Updated weights for policy 1, policy_version 23500 (0.0009) [2023-10-12 20:56:52,217][44959] Updated weights for policy 1, policy_version 23510 (0.0009) [2023-10-12 20:56:52,491][44958] Updated weights for policy 0, policy_version 23370 (0.0008) [2023-10-12 20:56:52,582][44959] Updated weights for policy 1, policy_version 23520 (0.0008) [2023-10-12 20:56:52,869][44958] Updated weights for policy 0, policy_version 23380 (0.0007) [2023-10-12 20:56:53,243][44958] Updated weights for policy 0, policy_version 23390 (0.0009) [2023-10-12 20:56:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48037888. Throughput: 0: 1639.7, 1: 1637.2. Samples: 12021640. Policy #0 lag: (min: 27.0, avg: 34.8, max: 59.0) [2023-10-12 20:56:56,443][43579] Avg episode reward: [(0, '265.640'), (1, '283.950')] [2023-10-12 20:56:56,922][44959] Updated weights for policy 1, policy_version 23530 (0.0010) [2023-10-12 20:56:57,253][44958] Updated weights for policy 0, policy_version 23400 (0.0009) [2023-10-12 20:56:57,293][44959] Updated weights for policy 1, policy_version 23540 (0.0009) [2023-10-12 20:56:57,635][44958] Updated weights for policy 0, policy_version 23410 (0.0008) [2023-10-12 20:56:57,662][44959] Updated weights for policy 1, policy_version 23550 (0.0007) [2023-10-12 20:56:57,998][44958] Updated weights for policy 0, policy_version 23420 (0.0009) [2023-10-12 20:57:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48103424. Throughput: 0: 1631.4, 1: 1641.8. Samples: 12041874. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 20:57:01,443][43579] Avg episode reward: [(0, '268.530'), (1, '282.290')] [2023-10-12 20:57:01,793][44959] Updated weights for policy 1, policy_version 23560 (0.0009) [2023-10-12 20:57:02,156][44959] Updated weights for policy 1, policy_version 23570 (0.0008) [2023-10-12 20:57:02,264][44958] Updated weights for policy 0, policy_version 23430 (0.0007) [2023-10-12 20:57:02,528][44959] Updated weights for policy 1, policy_version 23580 (0.0009) [2023-10-12 20:57:02,648][44958] Updated weights for policy 0, policy_version 23440 (0.0008) [2023-10-12 20:57:03,020][44958] Updated weights for policy 0, policy_version 23450 (0.0009) [2023-10-12 20:57:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48168960. Throughput: 0: 1631.6, 1: 1641.6. Samples: 12050582. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 20:57:06,444][43579] Avg episode reward: [(0, '267.390'), (1, '284.840')] [2023-10-12 20:57:06,617][44959] Updated weights for policy 1, policy_version 23590 (0.0008) [2023-10-12 20:57:06,988][44959] Updated weights for policy 1, policy_version 23600 (0.0010) [2023-10-12 20:57:07,102][44958] Updated weights for policy 0, policy_version 23460 (0.0008) [2023-10-12 20:57:07,358][44959] Updated weights for policy 1, policy_version 23610 (0.0009) [2023-10-12 20:57:07,482][44958] Updated weights for policy 0, policy_version 23470 (0.0008) [2023-10-12 20:57:07,864][44958] Updated weights for policy 0, policy_version 23480 (0.0008) [2023-10-12 20:57:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48234496. Throughput: 0: 1638.1, 1: 1642.4. Samples: 12070782. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 20:57:11,443][43579] Avg episode reward: [(0, '272.040'), (1, '282.750')] [2023-10-12 20:57:11,632][44959] Updated weights for policy 1, policy_version 23620 (0.0009) [2023-10-12 20:57:11,996][44959] Updated weights for policy 1, policy_version 23630 (0.0009) [2023-10-12 20:57:12,085][44958] Updated weights for policy 0, policy_version 23490 (0.0009) [2023-10-12 20:57:12,361][44959] Updated weights for policy 1, policy_version 23640 (0.0008) [2023-10-12 20:57:12,447][44958] Updated weights for policy 0, policy_version 23500 (0.0007) [2023-10-12 20:57:12,827][44958] Updated weights for policy 0, policy_version 23510 (0.0008) [2023-10-12 20:57:13,206][44958] Updated weights for policy 0, policy_version 23520 (0.0007) [2023-10-12 20:57:16,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48300032. Throughput: 0: 1637.5, 1: 1642.6. Samples: 12091012. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-12 20:57:16,443][43579] Avg episode reward: [(0, '267.920'), (1, '284.830')] [2023-10-12 20:57:16,575][44959] Updated weights for policy 1, policy_version 23650 (0.0008) [2023-10-12 20:57:16,960][44959] Updated weights for policy 1, policy_version 23660 (0.0010) [2023-10-12 20:57:17,292][44958] Updated weights for policy 0, policy_version 23530 (0.0007) [2023-10-12 20:57:17,330][44959] Updated weights for policy 1, policy_version 23670 (0.0008) [2023-10-12 20:57:17,663][44958] Updated weights for policy 0, policy_version 23540 (0.0008) [2023-10-12 20:57:17,687][44959] Updated weights for policy 1, policy_version 23680 (0.0008) [2023-10-12 20:57:18,042][44958] Updated weights for policy 0, policy_version 23550 (0.0010) [2023-10-12 20:57:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48365568. Throughput: 0: 1637.2, 1: 1643.8. Samples: 12099702. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 20:57:21,443][43579] Avg episode reward: [(0, '269.400'), (1, '279.580')] [2023-10-12 20:57:21,840][44959] Updated weights for policy 1, policy_version 23690 (0.0009) [2023-10-12 20:57:22,208][44959] Updated weights for policy 1, policy_version 23700 (0.0010) [2023-10-12 20:57:22,441][44958] Updated weights for policy 0, policy_version 23560 (0.0009) [2023-10-12 20:57:22,580][44959] Updated weights for policy 1, policy_version 23710 (0.0008) [2023-10-12 20:57:22,821][44958] Updated weights for policy 0, policy_version 23570 (0.0009) [2023-10-12 20:57:23,182][44958] Updated weights for policy 0, policy_version 23580 (0.0007) [2023-10-12 20:57:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48431104. Throughput: 0: 1630.5, 1: 1647.3. Samples: 12119788. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 20:57:26,444][43579] Avg episode reward: [(0, '268.200'), (1, '277.380')] [2023-10-12 20:57:26,677][44959] Updated weights for policy 1, policy_version 23720 (0.0010) [2023-10-12 20:57:27,035][44959] Updated weights for policy 1, policy_version 23730 (0.0007) [2023-10-12 20:57:27,401][44959] Updated weights for policy 1, policy_version 23740 (0.0007) [2023-10-12 20:57:27,518][44958] Updated weights for policy 0, policy_version 23590 (0.0007) [2023-10-12 20:57:27,889][44958] Updated weights for policy 0, policy_version 23600 (0.0007) [2023-10-12 20:57:28,267][44958] Updated weights for policy 0, policy_version 23610 (0.0010) [2023-10-12 20:57:31,443][44959] Updated weights for policy 1, policy_version 23750 (0.0008) [2023-10-12 20:57:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48496640. Throughput: 0: 1621.1, 1: 1649.6. Samples: 12139890. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 20:57:31,443][43579] Avg episode reward: [(0, '265.090'), (1, '279.840')] [2023-10-12 20:57:31,809][44959] Updated weights for policy 1, policy_version 23760 (0.0009) [2023-10-12 20:57:32,177][44959] Updated weights for policy 1, policy_version 23770 (0.0009) [2023-10-12 20:57:32,417][44958] Updated weights for policy 0, policy_version 23620 (0.0009) [2023-10-12 20:57:32,792][44958] Updated weights for policy 0, policy_version 23630 (0.0010) [2023-10-12 20:57:33,165][44958] Updated weights for policy 0, policy_version 23640 (0.0010) [2023-10-12 20:57:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48562176. Throughput: 0: 1625.6, 1: 1643.0. Samples: 12148696. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 20:57:36,443][43579] Avg episode reward: [(0, '263.180'), (1, '278.220')] [2023-10-12 20:57:36,532][44959] Updated weights for policy 1, policy_version 23780 (0.0010) [2023-10-12 20:57:36,903][44959] Updated weights for policy 1, policy_version 23790 (0.0008) [2023-10-12 20:57:37,270][44959] Updated weights for policy 1, policy_version 23800 (0.0009) [2023-10-12 20:57:37,376][44958] Updated weights for policy 0, policy_version 23650 (0.0010) [2023-10-12 20:57:37,739][44958] Updated weights for policy 0, policy_version 23660 (0.0009) [2023-10-12 20:57:38,117][44958] Updated weights for policy 0, policy_version 23670 (0.0007) [2023-10-12 20:57:38,504][44958] Updated weights for policy 0, policy_version 23680 (0.0009) [2023-10-12 20:57:41,347][44959] Updated weights for policy 1, policy_version 23810 (0.0009) [2023-10-12 20:57:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48627712. Throughput: 0: 1628.8, 1: 1646.1. Samples: 12169014. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 20:57:41,444][43579] Avg episode reward: [(0, '261.410'), (1, '274.160')] [2023-10-12 20:57:41,713][44959] Updated weights for policy 1, policy_version 23820 (0.0009) [2023-10-12 20:57:42,085][44959] Updated weights for policy 1, policy_version 23830 (0.0010) [2023-10-12 20:57:42,453][44959] Updated weights for policy 1, policy_version 23840 (0.0007) [2023-10-12 20:57:42,519][44958] Updated weights for policy 0, policy_version 23690 (0.0007) [2023-10-12 20:57:42,889][44958] Updated weights for policy 0, policy_version 23700 (0.0007) [2023-10-12 20:57:43,260][44958] Updated weights for policy 0, policy_version 23710 (0.0007) [2023-10-12 20:57:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48693248. Throughput: 0: 1631.2, 1: 1642.0. Samples: 12189172. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 20:57:46,443][43579] Avg episode reward: [(0, '267.370'), (1, '272.740')] [2023-10-12 20:57:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000023712_24281088.pth... [2023-10-12 20:57:46,495][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000022176_22708224.pth [2023-10-12 20:57:46,724][44959] Updated weights for policy 1, policy_version 23850 (0.0008) [2023-10-12 20:57:47,091][44959] Updated weights for policy 1, policy_version 23860 (0.0009) [2023-10-12 20:57:47,468][44959] Updated weights for policy 1, policy_version 23870 (0.0008) [2023-10-12 20:57:47,535][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000023872_24444928.pth... [2023-10-12 20:57:47,571][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000022304_22839296.pth [2023-10-12 20:57:47,591][44958] Updated weights for policy 0, policy_version 23720 (0.0009) [2023-10-12 20:57:47,964][44958] Updated weights for policy 0, policy_version 23730 (0.0008) [2023-10-12 20:57:48,335][44958] Updated weights for policy 0, policy_version 23740 (0.0011) [2023-10-12 20:57:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48758784. Throughput: 0: 1631.7, 1: 1641.8. Samples: 12197890. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 20:57:51,443][43579] Avg episode reward: [(0, '269.920'), (1, '275.780')] [2023-10-12 20:57:51,637][44959] Updated weights for policy 1, policy_version 23880 (0.0008) [2023-10-12 20:57:52,006][44959] Updated weights for policy 1, policy_version 23890 (0.0009) [2023-10-12 20:57:52,376][44959] Updated weights for policy 1, policy_version 23900 (0.0009) [2023-10-12 20:57:52,469][44958] Updated weights for policy 0, policy_version 23750 (0.0009) [2023-10-12 20:57:52,844][44958] Updated weights for policy 0, policy_version 23760 (0.0008) [2023-10-12 20:57:53,216][44958] Updated weights for policy 0, policy_version 23770 (0.0009) [2023-10-12 20:57:56,403][44959] Updated weights for policy 1, policy_version 23910 (0.0008) [2023-10-12 20:57:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48824320. Throughput: 0: 1628.8, 1: 1647.0. Samples: 12218194. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 20:57:56,443][43579] Avg episode reward: [(0, '267.790'), (1, '277.480')] [2023-10-12 20:57:56,778][44959] Updated weights for policy 1, policy_version 23920 (0.0010) [2023-10-12 20:57:57,139][44959] Updated weights for policy 1, policy_version 23930 (0.0008) [2023-10-12 20:57:57,343][44958] Updated weights for policy 0, policy_version 23780 (0.0008) [2023-10-12 20:57:57,720][44958] Updated weights for policy 0, policy_version 23790 (0.0007) [2023-10-12 20:57:58,090][44958] Updated weights for policy 0, policy_version 23800 (0.0007) [2023-10-12 20:58:01,413][44959] Updated weights for policy 1, policy_version 23940 (0.0008) [2023-10-12 20:58:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48889856. Throughput: 0: 1629.5, 1: 1651.2. Samples: 12238642. Policy #0 lag: (min: 10.0, avg: 25.3, max: 42.0) [2023-10-12 20:58:01,443][43579] Avg episode reward: [(0, '262.380'), (1, '276.470')] [2023-10-12 20:58:01,790][44959] Updated weights for policy 1, policy_version 23950 (0.0009) [2023-10-12 20:58:02,155][44959] Updated weights for policy 1, policy_version 23960 (0.0009) [2023-10-12 20:58:02,362][44958] Updated weights for policy 0, policy_version 23810 (0.0011) [2023-10-12 20:58:02,728][44958] Updated weights for policy 0, policy_version 23820 (0.0008) [2023-10-12 20:58:03,099][44958] Updated weights for policy 0, policy_version 23830 (0.0009) [2023-10-12 20:58:03,466][44958] Updated weights for policy 0, policy_version 23840 (0.0008) [2023-10-12 20:58:06,364][44959] Updated weights for policy 1, policy_version 23970 (0.0008) [2023-10-12 20:58:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 48955392. Throughput: 0: 1630.6, 1: 1653.2. Samples: 12247474. Policy #0 lag: (min: 10.0, avg: 25.3, max: 42.0) [2023-10-12 20:58:06,444][43579] Avg episode reward: [(0, '264.860'), (1, '278.010')] [2023-10-12 20:58:06,788][44959] Updated weights for policy 1, policy_version 23980 (0.0007) [2023-10-12 20:58:07,157][44959] Updated weights for policy 1, policy_version 23990 (0.0010) [2023-10-12 20:58:07,521][44959] Updated weights for policy 1, policy_version 24000 (0.0009) [2023-10-12 20:58:07,556][44958] Updated weights for policy 0, policy_version 23850 (0.0010) [2023-10-12 20:58:07,926][44958] Updated weights for policy 0, policy_version 23860 (0.0011) [2023-10-12 20:58:08,294][44958] Updated weights for policy 0, policy_version 23870 (0.0010) [2023-10-12 20:58:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49020928. Throughput: 0: 1635.9, 1: 1655.6. Samples: 12267906. Policy #0 lag: (min: 10.0, avg: 25.3, max: 42.0) [2023-10-12 20:58:11,444][43579] Avg episode reward: [(0, '260.920'), (1, '277.820')] [2023-10-12 20:58:11,617][44959] Updated weights for policy 1, policy_version 24010 (0.0009) [2023-10-12 20:58:11,979][44959] Updated weights for policy 1, policy_version 24020 (0.0010) [2023-10-12 20:58:12,351][44959] Updated weights for policy 1, policy_version 24030 (0.0008) [2023-10-12 20:58:12,448][44958] Updated weights for policy 0, policy_version 23880 (0.0009) [2023-10-12 20:58:12,814][44958] Updated weights for policy 0, policy_version 23890 (0.0009) [2023-10-12 20:58:13,188][44958] Updated weights for policy 0, policy_version 23900 (0.0010) [2023-10-12 20:58:16,370][44959] Updated weights for policy 1, policy_version 24040 (0.0010) [2023-10-12 20:58:16,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49086464. Throughput: 0: 1645.5, 1: 1648.5. Samples: 12288120. Policy #0 lag: (min: 10.0, avg: 25.3, max: 42.0) [2023-10-12 20:58:16,443][43579] Avg episode reward: [(0, '256.960'), (1, '277.740')] [2023-10-12 20:58:16,737][44959] Updated weights for policy 1, policy_version 24050 (0.0007) [2023-10-12 20:58:17,107][44959] Updated weights for policy 1, policy_version 24060 (0.0008) [2023-10-12 20:58:17,315][44958] Updated weights for policy 0, policy_version 23910 (0.0009) [2023-10-12 20:58:17,683][44958] Updated weights for policy 0, policy_version 23920 (0.0007) [2023-10-12 20:58:18,055][44958] Updated weights for policy 0, policy_version 23930 (0.0008) [2023-10-12 20:58:21,218][44959] Updated weights for policy 1, policy_version 24070 (0.0009) [2023-10-12 20:58:21,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49152000. Throughput: 0: 1645.9, 1: 1648.4. Samples: 12296938. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-12 20:58:21,443][43579] Avg episode reward: [(0, '252.160'), (1, '279.000')] [2023-10-12 20:58:21,598][44959] Updated weights for policy 1, policy_version 24080 (0.0009) [2023-10-12 20:58:21,963][44959] Updated weights for policy 1, policy_version 24090 (0.0010) [2023-10-12 20:58:22,424][44958] Updated weights for policy 0, policy_version 23940 (0.0009) [2023-10-12 20:58:22,785][44958] Updated weights for policy 0, policy_version 23950 (0.0009) [2023-10-12 20:58:23,157][44958] Updated weights for policy 0, policy_version 23960 (0.0008) [2023-10-12 20:58:25,945][44959] Updated weights for policy 1, policy_version 24100 (0.0009) [2023-10-12 20:58:26,314][44959] Updated weights for policy 1, policy_version 24110 (0.0009) [2023-10-12 20:58:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49217536. Throughput: 0: 1640.2, 1: 1649.6. Samples: 12317056. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-12 20:58:26,444][43579] Avg episode reward: [(0, '261.390'), (1, '274.660')] [2023-10-12 20:58:26,675][44959] Updated weights for policy 1, policy_version 24120 (0.0011) [2023-10-12 20:58:27,167][44958] Updated weights for policy 0, policy_version 23970 (0.0009) [2023-10-12 20:58:27,544][44958] Updated weights for policy 0, policy_version 23980 (0.0008) [2023-10-12 20:58:27,912][44958] Updated weights for policy 0, policy_version 23990 (0.0007) [2023-10-12 20:58:28,294][44958] Updated weights for policy 0, policy_version 24000 (0.0008) [2023-10-12 20:58:31,045][44959] Updated weights for policy 1, policy_version 24130 (0.0010) [2023-10-12 20:58:31,418][44959] Updated weights for policy 1, policy_version 24140 (0.0009) [2023-10-12 20:58:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49283072. Throughput: 0: 1646.2, 1: 1642.5. Samples: 12337166. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-12 20:58:31,444][43579] Avg episode reward: [(0, '258.150'), (1, '275.670')] [2023-10-12 20:58:31,779][44959] Updated weights for policy 1, policy_version 24150 (0.0011) [2023-10-12 20:58:32,150][44959] Updated weights for policy 1, policy_version 24160 (0.0010) [2023-10-12 20:58:32,506][44958] Updated weights for policy 0, policy_version 24010 (0.0011) [2023-10-12 20:58:32,880][44958] Updated weights for policy 0, policy_version 24020 (0.0011) [2023-10-12 20:58:33,254][44958] Updated weights for policy 0, policy_version 24030 (0.0007) [2023-10-12 20:58:36,396][44959] Updated weights for policy 1, policy_version 24170 (0.0010) [2023-10-12 20:58:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49348608. Throughput: 0: 1648.3, 1: 1647.5. Samples: 12346202. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-12 20:58:36,444][43579] Avg episode reward: [(0, '259.790'), (1, '273.440')] [2023-10-12 20:58:36,774][44959] Updated weights for policy 1, policy_version 24180 (0.0010) [2023-10-12 20:58:37,143][44959] Updated weights for policy 1, policy_version 24190 (0.0008) [2023-10-12 20:58:37,305][44958] Updated weights for policy 0, policy_version 24040 (0.0010) [2023-10-12 20:58:37,683][44958] Updated weights for policy 0, policy_version 24050 (0.0009) [2023-10-12 20:58:38,043][44958] Updated weights for policy 0, policy_version 24060 (0.0008) [2023-10-12 20:58:41,316][44959] Updated weights for policy 1, policy_version 24200 (0.0010) [2023-10-12 20:58:41,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49414144. Throughput: 0: 1649.5, 1: 1646.3. Samples: 12366502. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-12 20:58:41,443][43579] Avg episode reward: [(0, '262.390'), (1, '273.380')] [2023-10-12 20:58:41,683][44959] Updated weights for policy 1, policy_version 24210 (0.0007) [2023-10-12 20:58:42,017][44958] Updated weights for policy 0, policy_version 24070 (0.0009) [2023-10-12 20:58:42,043][44959] Updated weights for policy 1, policy_version 24220 (0.0007) [2023-10-12 20:58:42,392][44958] Updated weights for policy 0, policy_version 24080 (0.0008) [2023-10-12 20:58:42,767][44958] Updated weights for policy 0, policy_version 24090 (0.0010) [2023-10-12 20:58:46,119][44959] Updated weights for policy 1, policy_version 24230 (0.0009) [2023-10-12 20:58:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49479680. Throughput: 0: 1645.4, 1: 1641.9. Samples: 12386568. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-12 20:58:46,443][43579] Avg episode reward: [(0, '267.030'), (1, '273.280')] [2023-10-12 20:58:46,485][44959] Updated weights for policy 1, policy_version 24240 (0.0011) [2023-10-12 20:58:46,846][44959] Updated weights for policy 1, policy_version 24250 (0.0008) [2023-10-12 20:58:47,277][44958] Updated weights for policy 0, policy_version 24100 (0.0010) [2023-10-12 20:58:47,649][44958] Updated weights for policy 0, policy_version 24110 (0.0010) [2023-10-12 20:58:48,021][44958] Updated weights for policy 0, policy_version 24120 (0.0009) [2023-10-12 20:58:51,180][44959] Updated weights for policy 1, policy_version 24260 (0.0010) [2023-10-12 20:58:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49545216. Throughput: 0: 1647.2, 1: 1644.9. Samples: 12395620. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-12 20:58:51,443][43579] Avg episode reward: [(0, '267.630'), (1, '277.320')] [2023-10-12 20:58:51,575][44959] Updated weights for policy 1, policy_version 24270 (0.0010) [2023-10-12 20:58:51,950][44959] Updated weights for policy 1, policy_version 24280 (0.0009) [2023-10-12 20:58:52,263][44958] Updated weights for policy 0, policy_version 24130 (0.0008) [2023-10-12 20:58:52,636][44958] Updated weights for policy 0, policy_version 24140 (0.0008) [2023-10-12 20:58:53,011][44958] Updated weights for policy 0, policy_version 24150 (0.0009) [2023-10-12 20:58:53,383][44958] Updated weights for policy 0, policy_version 24160 (0.0010) [2023-10-12 20:58:55,993][44959] Updated weights for policy 1, policy_version 24290 (0.0008) [2023-10-12 20:58:56,365][44959] Updated weights for policy 1, policy_version 24300 (0.0009) [2023-10-12 20:58:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49610752. Throughput: 0: 1649.6, 1: 1641.2. Samples: 12415992. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-12 20:58:56,444][43579] Avg episode reward: [(0, '260.990'), (1, '276.670')] [2023-10-12 20:58:56,736][44959] Updated weights for policy 1, policy_version 24310 (0.0008) [2023-10-12 20:58:57,106][44959] Updated weights for policy 1, policy_version 24320 (0.0008) [2023-10-12 20:58:57,419][44958] Updated weights for policy 0, policy_version 24170 (0.0008) [2023-10-12 20:58:57,785][44958] Updated weights for policy 0, policy_version 24180 (0.0008) [2023-10-12 20:58:58,157][44958] Updated weights for policy 0, policy_version 24190 (0.0009) [2023-10-12 20:59:01,341][44959] Updated weights for policy 1, policy_version 24330 (0.0007) [2023-10-12 20:59:01,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 49676288. Throughput: 0: 1647.8, 1: 1641.9. Samples: 12436154. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-12 20:59:01,444][43579] Avg episode reward: [(0, '262.550'), (1, '278.270')] [2023-10-12 20:59:01,707][44959] Updated weights for policy 1, policy_version 24340 (0.0010) [2023-10-12 20:59:02,076][44959] Updated weights for policy 1, policy_version 24350 (0.0009) [2023-10-12 20:59:02,300][44958] Updated weights for policy 0, policy_version 24200 (0.0007) [2023-10-12 20:59:02,666][44958] Updated weights for policy 0, policy_version 24210 (0.0010) [2023-10-12 20:59:03,051][44958] Updated weights for policy 0, policy_version 24220 (0.0010) [2023-10-12 20:59:06,101][44959] Updated weights for policy 1, policy_version 24360 (0.0009) [2023-10-12 20:59:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49741824. Throughput: 0: 1643.3, 1: 1650.0. Samples: 12445140. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-12 20:59:06,443][43579] Avg episode reward: [(0, '260.940'), (1, '278.190')] [2023-10-12 20:59:06,460][44959] Updated weights for policy 1, policy_version 24370 (0.0011) [2023-10-12 20:59:06,833][44959] Updated weights for policy 1, policy_version 24380 (0.0010) [2023-10-12 20:59:07,248][44958] Updated weights for policy 0, policy_version 24230 (0.0008) [2023-10-12 20:59:07,619][44958] Updated weights for policy 0, policy_version 24240 (0.0009) [2023-10-12 20:59:07,989][44958] Updated weights for policy 0, policy_version 24250 (0.0007) [2023-10-12 20:59:10,995][44959] Updated weights for policy 1, policy_version 24390 (0.0008) [2023-10-12 20:59:11,361][44959] Updated weights for policy 1, policy_version 24400 (0.0007) [2023-10-12 20:59:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49807360. Throughput: 0: 1649.3, 1: 1652.3. Samples: 12465630. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-12 20:59:11,443][43579] Avg episode reward: [(0, '258.770'), (1, '283.140')] [2023-10-12 20:59:11,739][44959] Updated weights for policy 1, policy_version 24410 (0.0009) [2023-10-12 20:59:12,196][44958] Updated weights for policy 0, policy_version 24260 (0.0008) [2023-10-12 20:59:12,569][44958] Updated weights for policy 0, policy_version 24270 (0.0010) [2023-10-12 20:59:12,932][44958] Updated weights for policy 0, policy_version 24280 (0.0010) [2023-10-12 20:59:15,658][44959] Updated weights for policy 1, policy_version 24420 (0.0010) [2023-10-12 20:59:16,026][44959] Updated weights for policy 1, policy_version 24430 (0.0009) [2023-10-12 20:59:16,395][44959] Updated weights for policy 1, policy_version 24440 (0.0009) [2023-10-12 20:59:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49872896. Throughput: 0: 1640.4, 1: 1650.1. Samples: 12485240. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-12 20:59:16,443][43579] Avg episode reward: [(0, '256.220'), (1, '280.570')] [2023-10-12 20:59:17,119][44958] Updated weights for policy 0, policy_version 24290 (0.0010) [2023-10-12 20:59:17,493][44958] Updated weights for policy 0, policy_version 24300 (0.0010) [2023-10-12 20:59:17,862][44958] Updated weights for policy 0, policy_version 24310 (0.0008) [2023-10-12 20:59:18,239][44958] Updated weights for policy 0, policy_version 24320 (0.0009) [2023-10-12 20:59:20,614][44959] Updated weights for policy 1, policy_version 24450 (0.0007) [2023-10-12 20:59:20,982][44959] Updated weights for policy 1, policy_version 24460 (0.0008) [2023-10-12 20:59:21,339][44959] Updated weights for policy 1, policy_version 24470 (0.0009) [2023-10-12 20:59:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 49938432. Throughput: 0: 1638.1, 1: 1660.6. Samples: 12494640. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) [2023-10-12 20:59:21,443][43579] Avg episode reward: [(0, '257.570'), (1, '273.300')] [2023-10-12 20:59:21,710][44959] Updated weights for policy 1, policy_version 24480 (0.0008) [2023-10-12 20:59:22,390][44958] Updated weights for policy 0, policy_version 24330 (0.0010) [2023-10-12 20:59:22,765][44958] Updated weights for policy 0, policy_version 24340 (0.0011) [2023-10-12 20:59:23,136][44958] Updated weights for policy 0, policy_version 24350 (0.0011) [2023-10-12 20:59:25,840][44959] Updated weights for policy 1, policy_version 24490 (0.0007) [2023-10-12 20:59:26,204][44959] Updated weights for policy 1, policy_version 24500 (0.0009) [2023-10-12 20:59:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 50003968. Throughput: 0: 1635.6, 1: 1658.8. Samples: 12514750. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) [2023-10-12 20:59:26,443][43579] Avg episode reward: [(0, '257.360'), (1, '273.530')] [2023-10-12 20:59:26,576][44959] Updated weights for policy 1, policy_version 24510 (0.0008) [2023-10-12 20:59:27,438][44958] Updated weights for policy 0, policy_version 24360 (0.0007) [2023-10-12 20:59:27,806][44958] Updated weights for policy 0, policy_version 24370 (0.0009) [2023-10-12 20:59:28,179][44958] Updated weights for policy 0, policy_version 24380 (0.0010) [2023-10-12 20:59:30,912][44959] Updated weights for policy 1, policy_version 24520 (0.0008) [2023-10-12 20:59:31,284][44959] Updated weights for policy 1, policy_version 24530 (0.0007) [2023-10-12 20:59:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 50069504. Throughput: 0: 1637.8, 1: 1654.8. Samples: 12534732. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) [2023-10-12 20:59:31,443][43579] Avg episode reward: [(0, '256.000'), (1, '268.410')] [2023-10-12 20:59:31,653][44959] Updated weights for policy 1, policy_version 24540 (0.0007) [2023-10-12 20:59:32,366][44958] Updated weights for policy 0, policy_version 24390 (0.0011) [2023-10-12 20:59:32,739][44958] Updated weights for policy 0, policy_version 24400 (0.0009) [2023-10-12 20:59:33,109][44958] Updated weights for policy 0, policy_version 24410 (0.0008) [2023-10-12 20:59:35,514][44959] Updated weights for policy 1, policy_version 24550 (0.0007) [2023-10-12 20:59:35,880][44959] Updated weights for policy 1, policy_version 24560 (0.0008) [2023-10-12 20:59:36,250][44959] Updated weights for policy 1, policy_version 24570 (0.0008) [2023-10-12 20:59:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 50135040. Throughput: 0: 1636.6, 1: 1663.9. Samples: 12544140. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) [2023-10-12 20:59:36,443][43579] Avg episode reward: [(0, '258.440'), (1, '269.370')] [2023-10-12 20:59:37,234][44958] Updated weights for policy 0, policy_version 24420 (0.0008) [2023-10-12 20:59:37,611][44958] Updated weights for policy 0, policy_version 24430 (0.0008) [2023-10-12 20:59:37,974][44958] Updated weights for policy 0, policy_version 24440 (0.0010) [2023-10-12 20:59:40,367][44959] Updated weights for policy 1, policy_version 24580 (0.0009) [2023-10-12 20:59:40,734][44959] Updated weights for policy 1, policy_version 24590 (0.0009) [2023-10-12 20:59:41,115][44959] Updated weights for policy 1, policy_version 24600 (0.0007) [2023-10-12 20:59:41,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50233344. Throughput: 0: 1633.3, 1: 1660.4. Samples: 12564210. Policy #0 lag: (min: 1.0, avg: 5.5, max: 33.0) [2023-10-12 20:59:41,443][43579] Avg episode reward: [(0, '262.180'), (1, '271.180')] [2023-10-12 20:59:42,261][44958] Updated weights for policy 0, policy_version 24450 (0.0009) [2023-10-12 20:59:42,647][44958] Updated weights for policy 0, policy_version 24460 (0.0011) [2023-10-12 20:59:43,011][44958] Updated weights for policy 0, policy_version 24470 (0.0008) [2023-10-12 20:59:43,382][44958] Updated weights for policy 0, policy_version 24480 (0.0008) [2023-10-12 20:59:45,047][44959] Updated weights for policy 1, policy_version 24610 (0.0009) [2023-10-12 20:59:45,417][44959] Updated weights for policy 1, policy_version 24620 (0.0007) [2023-10-12 20:59:45,781][44959] Updated weights for policy 1, policy_version 24630 (0.0008) [2023-10-12 20:59:46,143][44959] Updated weights for policy 1, policy_version 24640 (0.0008) [2023-10-12 20:59:46,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 50298880. Throughput: 0: 1635.2, 1: 1645.8. Samples: 12583800. Policy #0 lag: (min: 1.0, avg: 5.5, max: 33.0) [2023-10-12 20:59:46,443][43579] Avg episode reward: [(0, '263.810'), (1, '273.080')] [2023-10-12 20:59:46,453][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000024480_25067520.pth... [2023-10-12 20:59:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000024640_25231360.pth... [2023-10-12 20:59:46,490][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000022944_23494656.pth [2023-10-12 20:59:46,494][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000023104_23658496.pth [2023-10-12 20:59:47,731][44958] Updated weights for policy 0, policy_version 24490 (0.0009) [2023-10-12 20:59:48,103][44958] Updated weights for policy 0, policy_version 24500 (0.0008) [2023-10-12 20:59:48,476][44958] Updated weights for policy 0, policy_version 24510 (0.0009) [2023-10-12 20:59:50,382][44959] Updated weights for policy 1, policy_version 24650 (0.0009) [2023-10-12 20:59:50,750][44959] Updated weights for policy 1, policy_version 24660 (0.0007) [2023-10-12 20:59:51,124][44959] Updated weights for policy 1, policy_version 24670 (0.0007) [2023-10-12 20:59:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50364416. Throughput: 0: 1635.9, 1: 1664.1. Samples: 12593638. Policy #0 lag: (min: 1.0, avg: 5.5, max: 33.0) [2023-10-12 20:59:51,444][43579] Avg episode reward: [(0, '264.760'), (1, '276.650')] [2023-10-12 20:59:52,472][44958] Updated weights for policy 0, policy_version 24520 (0.0008) [2023-10-12 20:59:52,842][44958] Updated weights for policy 0, policy_version 24530 (0.0009) [2023-10-12 20:59:53,218][44958] Updated weights for policy 0, policy_version 24540 (0.0008) [2023-10-12 20:59:55,290][44959] Updated weights for policy 1, policy_version 24680 (0.0008) [2023-10-12 20:59:55,658][44959] Updated weights for policy 1, policy_version 24690 (0.0008) [2023-10-12 20:59:56,034][44959] Updated weights for policy 1, policy_version 24700 (0.0007) [2023-10-12 20:59:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 50429952. Throughput: 0: 1634.4, 1: 1661.6. Samples: 12613948. Policy #0 lag: (min: 1.0, avg: 5.5, max: 33.0) [2023-10-12 20:59:56,443][43579] Avg episode reward: [(0, '269.270'), (1, '281.300')] [2023-10-12 20:59:57,392][44958] Updated weights for policy 0, policy_version 24550 (0.0010) [2023-10-12 20:59:57,781][44958] Updated weights for policy 0, policy_version 24560 (0.0009) [2023-10-12 20:59:58,161][44958] Updated weights for policy 0, policy_version 24570 (0.0007) [2023-10-12 21:00:00,151][44959] Updated weights for policy 1, policy_version 24710 (0.0009) [2023-10-12 21:00:00,514][44959] Updated weights for policy 1, policy_version 24720 (0.0010) [2023-10-12 21:00:00,881][44959] Updated weights for policy 1, policy_version 24730 (0.0009) [2023-10-12 21:00:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50495488. Throughput: 0: 1637.5, 1: 1648.4. Samples: 12633104. Policy #0 lag: (min: 15.0, avg: 15.6, max: 32.0) [2023-10-12 21:00:01,444][43579] Avg episode reward: [(0, '272.610'), (1, '280.640')] [2023-10-12 21:00:02,330][44958] Updated weights for policy 0, policy_version 24580 (0.0010) [2023-10-12 21:00:02,723][44958] Updated weights for policy 0, policy_version 24590 (0.0008) [2023-10-12 21:00:03,101][44958] Updated weights for policy 0, policy_version 24600 (0.0009) [2023-10-12 21:00:05,266][44959] Updated weights for policy 1, policy_version 24740 (0.0011) [2023-10-12 21:00:05,640][44959] Updated weights for policy 1, policy_version 24750 (0.0008) [2023-10-12 21:00:06,005][44959] Updated weights for policy 1, policy_version 24760 (0.0008) [2023-10-12 21:00:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50561024. Throughput: 0: 1635.1, 1: 1660.3. Samples: 12642934. Policy #0 lag: (min: 15.0, avg: 15.6, max: 32.0) [2023-10-12 21:00:06,444][43579] Avg episode reward: [(0, '274.880'), (1, '270.040')] [2023-10-12 21:00:07,205][44958] Updated weights for policy 0, policy_version 24610 (0.0008) [2023-10-12 21:00:07,567][44958] Updated weights for policy 0, policy_version 24620 (0.0007) [2023-10-12 21:00:07,939][44958] Updated weights for policy 0, policy_version 24630 (0.0009) [2023-10-12 21:00:08,310][44958] Updated weights for policy 0, policy_version 24640 (0.0011) [2023-10-12 21:00:09,957][44959] Updated weights for policy 1, policy_version 24770 (0.0008) [2023-10-12 21:00:10,318][44959] Updated weights for policy 1, policy_version 24780 (0.0009) [2023-10-12 21:00:10,693][44959] Updated weights for policy 1, policy_version 24790 (0.0007) [2023-10-12 21:00:11,058][44959] Updated weights for policy 1, policy_version 24800 (0.0010) [2023-10-12 21:00:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50626560. Throughput: 0: 1635.2, 1: 1661.1. Samples: 12663086. Policy #0 lag: (min: 15.0, avg: 15.6, max: 32.0) [2023-10-12 21:00:11,443][43579] Avg episode reward: [(0, '276.710'), (1, '262.470')] [2023-10-12 21:00:12,605][44958] Updated weights for policy 0, policy_version 24650 (0.0009) [2023-10-12 21:00:12,981][44958] Updated weights for policy 0, policy_version 24660 (0.0009) [2023-10-12 21:00:13,357][44958] Updated weights for policy 0, policy_version 24670 (0.0009) [2023-10-12 21:00:15,317][44959] Updated weights for policy 1, policy_version 24810 (0.0009) [2023-10-12 21:00:15,682][44959] Updated weights for policy 1, policy_version 24820 (0.0007) [2023-10-12 21:00:16,056][44959] Updated weights for policy 1, policy_version 24830 (0.0007) [2023-10-12 21:00:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50692096. Throughput: 0: 1642.7, 1: 1642.9. Samples: 12682584. Policy #0 lag: (min: 15.0, avg: 15.6, max: 32.0) [2023-10-12 21:00:16,444][43579] Avg episode reward: [(0, '276.310'), (1, '263.320')] [2023-10-12 21:00:17,206][44958] Updated weights for policy 0, policy_version 24680 (0.0007) [2023-10-12 21:00:17,573][44958] Updated weights for policy 0, policy_version 24690 (0.0007) [2023-10-12 21:00:17,943][44958] Updated weights for policy 0, policy_version 24700 (0.0007) [2023-10-12 21:00:20,229][44959] Updated weights for policy 1, policy_version 24840 (0.0009) [2023-10-12 21:00:20,586][44959] Updated weights for policy 1, policy_version 24850 (0.0010) [2023-10-12 21:00:20,950][44959] Updated weights for policy 1, policy_version 24860 (0.0008) [2023-10-12 21:00:21,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50757632. Throughput: 0: 1640.2, 1: 1656.5. Samples: 12692494. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 21:00:21,444][43579] Avg episode reward: [(0, '278.890'), (1, '258.670')] [2023-10-12 21:00:22,274][44958] Updated weights for policy 0, policy_version 24710 (0.0008) [2023-10-12 21:00:22,646][44958] Updated weights for policy 0, policy_version 24720 (0.0011) [2023-10-12 21:00:23,023][44958] Updated weights for policy 0, policy_version 24730 (0.0007) [2023-10-12 21:00:25,216][44959] Updated weights for policy 1, policy_version 24870 (0.0009) [2023-10-12 21:00:25,600][44959] Updated weights for policy 1, policy_version 24880 (0.0008) [2023-10-12 21:00:25,964][44959] Updated weights for policy 1, policy_version 24890 (0.0007) [2023-10-12 21:00:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50823168. Throughput: 0: 1632.5, 1: 1659.4. Samples: 12712344. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 21:00:26,444][43579] Avg episode reward: [(0, '278.440'), (1, '262.070')] [2023-10-12 21:00:27,244][44958] Updated weights for policy 0, policy_version 24740 (0.0009) [2023-10-12 21:00:27,613][44958] Updated weights for policy 0, policy_version 24750 (0.0009) [2023-10-12 21:00:27,988][44958] Updated weights for policy 0, policy_version 24760 (0.0010) [2023-10-12 21:00:30,109][44959] Updated weights for policy 1, policy_version 24900 (0.0010) [2023-10-12 21:00:30,475][44959] Updated weights for policy 1, policy_version 24910 (0.0008) [2023-10-12 21:00:30,853][44959] Updated weights for policy 1, policy_version 24920 (0.0008) [2023-10-12 21:00:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50888704. Throughput: 0: 1631.9, 1: 1653.4. Samples: 12731636. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 21:00:31,444][43579] Avg episode reward: [(0, '277.660'), (1, '265.040')] [2023-10-12 21:00:32,273][44958] Updated weights for policy 0, policy_version 24770 (0.0009) [2023-10-12 21:00:32,653][44958] Updated weights for policy 0, policy_version 24780 (0.0008) [2023-10-12 21:00:33,034][44958] Updated weights for policy 0, policy_version 24790 (0.0008) [2023-10-12 21:00:33,400][44958] Updated weights for policy 0, policy_version 24800 (0.0008) [2023-10-12 21:00:35,129][44959] Updated weights for policy 1, policy_version 24930 (0.0007) [2023-10-12 21:00:35,501][44959] Updated weights for policy 1, policy_version 24940 (0.0008) [2023-10-12 21:00:35,869][44959] Updated weights for policy 1, policy_version 24950 (0.0007) [2023-10-12 21:00:36,239][44959] Updated weights for policy 1, policy_version 24960 (0.0007) [2023-10-12 21:00:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 50954240. Throughput: 0: 1633.6, 1: 1655.2. Samples: 12741634. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 21:00:36,444][43579] Avg episode reward: [(0, '276.430'), (1, '275.170')] [2023-10-12 21:00:37,672][44958] Updated weights for policy 0, policy_version 24810 (0.0008) [2023-10-12 21:00:38,047][44958] Updated weights for policy 0, policy_version 24820 (0.0008) [2023-10-12 21:00:38,414][44958] Updated weights for policy 0, policy_version 24830 (0.0007) [2023-10-12 21:00:40,285][44959] Updated weights for policy 1, policy_version 24970 (0.0007) [2023-10-12 21:00:40,649][44959] Updated weights for policy 1, policy_version 24980 (0.0010) [2023-10-12 21:00:41,030][44959] Updated weights for policy 1, policy_version 24990 (0.0008) [2023-10-12 21:00:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51019776. Throughput: 0: 1632.7, 1: 1653.2. Samples: 12761812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:00:41,443][43579] Avg episode reward: [(0, '281.270'), (1, '276.700')] [2023-10-12 21:00:41,444][44518] Saving new best policy, reward=281.270! [2023-10-12 21:00:42,379][44958] Updated weights for policy 0, policy_version 24840 (0.0008) [2023-10-12 21:00:42,744][44958] Updated weights for policy 0, policy_version 24850 (0.0007) [2023-10-12 21:00:43,124][44958] Updated weights for policy 0, policy_version 24860 (0.0008) [2023-10-12 21:00:45,084][44959] Updated weights for policy 1, policy_version 25000 (0.0008) [2023-10-12 21:00:45,449][44959] Updated weights for policy 1, policy_version 25010 (0.0007) [2023-10-12 21:00:45,815][44959] Updated weights for policy 1, policy_version 25020 (0.0008) [2023-10-12 21:00:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 51085312. Throughput: 0: 1638.7, 1: 1652.3. Samples: 12781196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:00:46,444][43579] Avg episode reward: [(0, '279.780'), (1, '278.800')] [2023-10-12 21:00:47,188][44958] Updated weights for policy 0, policy_version 24870 (0.0010) [2023-10-12 21:00:47,563][44958] Updated weights for policy 0, policy_version 24880 (0.0011) [2023-10-12 21:00:47,941][44958] Updated weights for policy 0, policy_version 24890 (0.0009) [2023-10-12 21:00:50,061][44959] Updated weights for policy 1, policy_version 25030 (0.0010) [2023-10-12 21:00:50,435][44959] Updated weights for policy 1, policy_version 25040 (0.0010) [2023-10-12 21:00:50,793][44959] Updated weights for policy 1, policy_version 25050 (0.0007) [2023-10-12 21:00:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51150848. Throughput: 0: 1642.4, 1: 1650.9. Samples: 12791132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:00:51,444][43579] Avg episode reward: [(0, '268.980'), (1, '281.830')] [2023-10-12 21:00:52,264][44958] Updated weights for policy 0, policy_version 24900 (0.0009) [2023-10-12 21:00:52,646][44958] Updated weights for policy 0, policy_version 24910 (0.0009) [2023-10-12 21:00:53,028][44958] Updated weights for policy 0, policy_version 24920 (0.0009) [2023-10-12 21:00:54,993][44959] Updated weights for policy 1, policy_version 25060 (0.0007) [2023-10-12 21:00:55,365][44959] Updated weights for policy 1, policy_version 25070 (0.0007) [2023-10-12 21:00:55,736][44959] Updated weights for policy 1, policy_version 25080 (0.0008) [2023-10-12 21:00:56,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51216384. Throughput: 0: 1641.8, 1: 1645.2. Samples: 12811000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:00:56,443][43579] Avg episode reward: [(0, '266.910'), (1, '281.290')] [2023-10-12 21:00:57,302][44958] Updated weights for policy 0, policy_version 24930 (0.0010) [2023-10-12 21:00:57,675][44958] Updated weights for policy 0, policy_version 24940 (0.0011) [2023-10-12 21:00:58,048][44958] Updated weights for policy 0, policy_version 24950 (0.0011) [2023-10-12 21:00:58,427][44958] Updated weights for policy 0, policy_version 24960 (0.0009) [2023-10-12 21:00:59,803][44959] Updated weights for policy 1, policy_version 25090 (0.0009) [2023-10-12 21:01:00,172][44959] Updated weights for policy 1, policy_version 25100 (0.0008) [2023-10-12 21:01:00,542][44959] Updated weights for policy 1, policy_version 25110 (0.0008) [2023-10-12 21:01:00,911][44959] Updated weights for policy 1, policy_version 25120 (0.0008) [2023-10-12 21:01:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51281920. Throughput: 0: 1637.7, 1: 1646.0. Samples: 12830352. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-12 21:01:01,444][43579] Avg episode reward: [(0, '268.830'), (1, '282.650')] [2023-10-12 21:01:02,523][44958] Updated weights for policy 0, policy_version 24970 (0.0010) [2023-10-12 21:01:02,902][44958] Updated weights for policy 0, policy_version 24980 (0.0008) [2023-10-12 21:01:03,267][44958] Updated weights for policy 0, policy_version 24990 (0.0010) [2023-10-12 21:01:05,030][44959] Updated weights for policy 1, policy_version 25130 (0.0007) [2023-10-12 21:01:05,397][44959] Updated weights for policy 1, policy_version 25140 (0.0007) [2023-10-12 21:01:05,760][44959] Updated weights for policy 1, policy_version 25150 (0.0010) [2023-10-12 21:01:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51347456. Throughput: 0: 1639.8, 1: 1649.0. Samples: 12840488. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-12 21:01:06,443][43579] Avg episode reward: [(0, '267.670'), (1, '279.530')] [2023-10-12 21:01:07,525][44958] Updated weights for policy 0, policy_version 25000 (0.0009) [2023-10-12 21:01:07,899][44958] Updated weights for policy 0, policy_version 25010 (0.0010) [2023-10-12 21:01:08,276][44958] Updated weights for policy 0, policy_version 25020 (0.0011) [2023-10-12 21:01:09,865][44959] Updated weights for policy 1, policy_version 25160 (0.0009) [2023-10-12 21:01:10,249][44959] Updated weights for policy 1, policy_version 25170 (0.0009) [2023-10-12 21:01:10,626][44959] Updated weights for policy 1, policy_version 25180 (0.0009) [2023-10-12 21:01:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 51412992. Throughput: 0: 1642.7, 1: 1638.5. Samples: 12859996. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-12 21:01:11,443][43579] Avg episode reward: [(0, '267.300'), (1, '275.380')] [2023-10-12 21:01:12,485][44958] Updated weights for policy 0, policy_version 25030 (0.0009) [2023-10-12 21:01:12,866][44958] Updated weights for policy 0, policy_version 25040 (0.0010) [2023-10-12 21:01:13,249][44958] Updated weights for policy 0, policy_version 25050 (0.0010) [2023-10-12 21:01:14,684][44959] Updated weights for policy 1, policy_version 25190 (0.0007) [2023-10-12 21:01:15,044][44959] Updated weights for policy 1, policy_version 25200 (0.0008) [2023-10-12 21:01:15,418][44959] Updated weights for policy 1, policy_version 25210 (0.0007) [2023-10-12 21:01:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51478528. Throughput: 0: 1643.7, 1: 1648.8. Samples: 12879802. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-12 21:01:16,443][43579] Avg episode reward: [(0, '272.320'), (1, '278.090')] [2023-10-12 21:01:17,411][44958] Updated weights for policy 0, policy_version 25060 (0.0010) [2023-10-12 21:01:17,776][44958] Updated weights for policy 0, policy_version 25070 (0.0009) [2023-10-12 21:01:18,153][44958] Updated weights for policy 0, policy_version 25080 (0.0008) [2023-10-12 21:01:19,571][44959] Updated weights for policy 1, policy_version 25220 (0.0007) [2023-10-12 21:01:19,937][44959] Updated weights for policy 1, policy_version 25230 (0.0008) [2023-10-12 21:01:20,297][44959] Updated weights for policy 1, policy_version 25240 (0.0008) [2023-10-12 21:01:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51544064. Throughput: 0: 1641.9, 1: 1652.1. Samples: 12889866. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:01:21,444][43579] Avg episode reward: [(0, '277.970'), (1, '277.270')] [2023-10-12 21:01:22,393][44958] Updated weights for policy 0, policy_version 25090 (0.0008) [2023-10-12 21:01:22,770][44958] Updated weights for policy 0, policy_version 25100 (0.0008) [2023-10-12 21:01:23,140][44958] Updated weights for policy 0, policy_version 25110 (0.0007) [2023-10-12 21:01:23,511][44958] Updated weights for policy 0, policy_version 25120 (0.0009) [2023-10-12 21:01:24,646][44959] Updated weights for policy 1, policy_version 25250 (0.0008) [2023-10-12 21:01:25,011][44959] Updated weights for policy 1, policy_version 25260 (0.0009) [2023-10-12 21:01:25,370][44959] Updated weights for policy 1, policy_version 25270 (0.0008) [2023-10-12 21:01:25,742][44959] Updated weights for policy 1, policy_version 25280 (0.0009) [2023-10-12 21:01:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51609600. Throughput: 0: 1642.2, 1: 1639.5. Samples: 12909488. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:01:26,443][43579] Avg episode reward: [(0, '279.470'), (1, '275.540')] [2023-10-12 21:01:27,524][44958] Updated weights for policy 0, policy_version 25130 (0.0008) [2023-10-12 21:01:27,902][44958] Updated weights for policy 0, policy_version 25140 (0.0007) [2023-10-12 21:01:28,271][44958] Updated weights for policy 0, policy_version 25150 (0.0008) [2023-10-12 21:01:29,767][44959] Updated weights for policy 1, policy_version 25290 (0.0009) [2023-10-12 21:01:30,137][44959] Updated weights for policy 1, policy_version 25300 (0.0007) [2023-10-12 21:01:30,502][44959] Updated weights for policy 1, policy_version 25310 (0.0008) [2023-10-12 21:01:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51675136. Throughput: 0: 1638.1, 1: 1646.7. Samples: 12929010. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:01:31,444][43579] Avg episode reward: [(0, '275.560'), (1, '276.700')] [2023-10-12 21:01:32,380][44958] Updated weights for policy 0, policy_version 25160 (0.0007) [2023-10-12 21:01:32,764][44958] Updated weights for policy 0, policy_version 25170 (0.0008) [2023-10-12 21:01:33,128][44958] Updated weights for policy 0, policy_version 25180 (0.0010) [2023-10-12 21:01:34,811][44959] Updated weights for policy 1, policy_version 25320 (0.0007) [2023-10-12 21:01:35,175][44959] Updated weights for policy 1, policy_version 25330 (0.0007) [2023-10-12 21:01:35,540][44959] Updated weights for policy 1, policy_version 25340 (0.0007) [2023-10-12 21:01:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51740672. Throughput: 0: 1634.1, 1: 1646.8. Samples: 12938772. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:01:36,444][43579] Avg episode reward: [(0, '273.480'), (1, '275.800')] [2023-10-12 21:01:37,193][44958] Updated weights for policy 0, policy_version 25190 (0.0009) [2023-10-12 21:01:37,566][44958] Updated weights for policy 0, policy_version 25200 (0.0007) [2023-10-12 21:01:37,934][44958] Updated weights for policy 0, policy_version 25210 (0.0009) [2023-10-12 21:01:39,631][44959] Updated weights for policy 1, policy_version 25350 (0.0007) [2023-10-12 21:01:39,994][44959] Updated weights for policy 1, policy_version 25360 (0.0007) [2023-10-12 21:01:40,360][44959] Updated weights for policy 1, policy_version 25370 (0.0008) [2023-10-12 21:01:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51806208. Throughput: 0: 1637.1, 1: 1642.4. Samples: 12958578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 21:01:41,443][43579] Avg episode reward: [(0, '274.220'), (1, '271.260')] [2023-10-12 21:01:42,202][44958] Updated weights for policy 0, policy_version 25220 (0.0008) [2023-10-12 21:01:42,580][44958] Updated weights for policy 0, policy_version 25230 (0.0007) [2023-10-12 21:01:42,949][44958] Updated weights for policy 0, policy_version 25240 (0.0009) [2023-10-12 21:01:44,570][44959] Updated weights for policy 1, policy_version 25380 (0.0010) [2023-10-12 21:01:44,934][44959] Updated weights for policy 1, policy_version 25390 (0.0009) [2023-10-12 21:01:45,309][44959] Updated weights for policy 1, policy_version 25400 (0.0011) [2023-10-12 21:01:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51871744. Throughput: 0: 1636.8, 1: 1649.5. Samples: 12978232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 21:01:46,444][43579] Avg episode reward: [(0, '275.250'), (1, '265.340')] [2023-10-12 21:01:46,457][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000025408_26017792.pth... [2023-10-12 21:01:46,457][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000025248_25853952.pth... [2023-10-12 21:01:46,498][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000023712_24281088.pth [2023-10-12 21:01:46,498][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000023872_24444928.pth [2023-10-12 21:01:47,265][44958] Updated weights for policy 0, policy_version 25250 (0.0007) [2023-10-12 21:01:47,648][44958] Updated weights for policy 0, policy_version 25260 (0.0007) [2023-10-12 21:01:48,006][44958] Updated weights for policy 0, policy_version 25270 (0.0010) [2023-10-12 21:01:48,387][44958] Updated weights for policy 0, policy_version 25280 (0.0009) [2023-10-12 21:01:49,480][44959] Updated weights for policy 1, policy_version 25410 (0.0009) [2023-10-12 21:01:49,840][44959] Updated weights for policy 1, policy_version 25420 (0.0009) [2023-10-12 21:01:50,206][44959] Updated weights for policy 1, policy_version 25430 (0.0010) [2023-10-12 21:01:50,581][44959] Updated weights for policy 1, policy_version 25440 (0.0009) [2023-10-12 21:01:51,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51937280. Throughput: 0: 1633.8, 1: 1646.3. Samples: 12988092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 21:01:51,444][43579] Avg episode reward: [(0, '274.090'), (1, '263.450')] [2023-10-12 21:01:52,450][44958] Updated weights for policy 0, policy_version 25290 (0.0009) [2023-10-12 21:01:52,816][44958] Updated weights for policy 0, policy_version 25300 (0.0011) [2023-10-12 21:01:53,195][44958] Updated weights for policy 0, policy_version 25310 (0.0010) [2023-10-12 21:01:54,758][44959] Updated weights for policy 1, policy_version 25450 (0.0007) [2023-10-12 21:01:55,121][44959] Updated weights for policy 1, policy_version 25460 (0.0007) [2023-10-12 21:01:55,499][44959] Updated weights for policy 1, policy_version 25470 (0.0007) [2023-10-12 21:01:56,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52002816. Throughput: 0: 1636.9, 1: 1641.5. Samples: 13007522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 21:01:56,443][43579] Avg episode reward: [(0, '274.470'), (1, '264.110')] [2023-10-12 21:01:57,450][44958] Updated weights for policy 0, policy_version 25320 (0.0010) [2023-10-12 21:01:57,817][44958] Updated weights for policy 0, policy_version 25330 (0.0010) [2023-10-12 21:01:58,201][44958] Updated weights for policy 0, policy_version 25340 (0.0007) [2023-10-12 21:01:59,526][44959] Updated weights for policy 1, policy_version 25480 (0.0008) [2023-10-12 21:01:59,898][44959] Updated weights for policy 1, policy_version 25490 (0.0010) [2023-10-12 21:02:00,285][44959] Updated weights for policy 1, policy_version 25500 (0.0009) [2023-10-12 21:02:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52068352. Throughput: 0: 1629.8, 1: 1645.0. Samples: 13027170. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:02:01,444][43579] Avg episode reward: [(0, '276.110'), (1, '256.730')] [2023-10-12 21:02:02,397][44958] Updated weights for policy 0, policy_version 25350 (0.0008) [2023-10-12 21:02:02,778][44958] Updated weights for policy 0, policy_version 25360 (0.0009) [2023-10-12 21:02:03,156][44958] Updated weights for policy 0, policy_version 25370 (0.0009) [2023-10-12 21:02:04,599][44959] Updated weights for policy 1, policy_version 25510 (0.0009) [2023-10-12 21:02:04,976][44959] Updated weights for policy 1, policy_version 25520 (0.0007) [2023-10-12 21:02:05,343][44959] Updated weights for policy 1, policy_version 25530 (0.0008) [2023-10-12 21:02:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52133888. Throughput: 0: 1629.6, 1: 1639.6. Samples: 13036984. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:02:06,443][43579] Avg episode reward: [(0, '274.690'), (1, '263.660')] [2023-10-12 21:02:07,452][44958] Updated weights for policy 0, policy_version 25380 (0.0009) [2023-10-12 21:02:07,830][44958] Updated weights for policy 0, policy_version 25390 (0.0009) [2023-10-12 21:02:08,208][44958] Updated weights for policy 0, policy_version 25400 (0.0008) [2023-10-12 21:02:09,547][44959] Updated weights for policy 1, policy_version 25540 (0.0008) [2023-10-12 21:02:09,917][44959] Updated weights for policy 1, policy_version 25550 (0.0007) [2023-10-12 21:02:10,273][44959] Updated weights for policy 1, policy_version 25560 (0.0007) [2023-10-12 21:02:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52199424. Throughput: 0: 1626.5, 1: 1641.2. Samples: 13056538. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:02:11,443][43579] Avg episode reward: [(0, '268.150'), (1, '260.490')] [2023-10-12 21:02:12,395][44958] Updated weights for policy 0, policy_version 25410 (0.0009) [2023-10-12 21:02:12,772][44958] Updated weights for policy 0, policy_version 25420 (0.0009) [2023-10-12 21:02:13,149][44958] Updated weights for policy 0, policy_version 25430 (0.0010) [2023-10-12 21:02:13,525][44958] Updated weights for policy 0, policy_version 25440 (0.0007) [2023-10-12 21:02:14,407][44959] Updated weights for policy 1, policy_version 25570 (0.0007) [2023-10-12 21:02:14,782][44959] Updated weights for policy 1, policy_version 25580 (0.0007) [2023-10-12 21:02:15,154][44959] Updated weights for policy 1, policy_version 25590 (0.0007) [2023-10-12 21:02:15,528][44959] Updated weights for policy 1, policy_version 25600 (0.0009) [2023-10-12 21:02:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52264960. Throughput: 0: 1625.6, 1: 1648.4. Samples: 13076338. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:02:16,444][43579] Avg episode reward: [(0, '266.440'), (1, '259.860')] [2023-10-12 21:02:17,715][44958] Updated weights for policy 0, policy_version 25450 (0.0007) [2023-10-12 21:02:18,090][44958] Updated weights for policy 0, policy_version 25460 (0.0008) [2023-10-12 21:02:18,466][44958] Updated weights for policy 0, policy_version 25470 (0.0008) [2023-10-12 21:02:19,691][44959] Updated weights for policy 1, policy_version 25610 (0.0010) [2023-10-12 21:02:20,067][44959] Updated weights for policy 1, policy_version 25620 (0.0010) [2023-10-12 21:02:20,430][44959] Updated weights for policy 1, policy_version 25630 (0.0010) [2023-10-12 21:02:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52330496. Throughput: 0: 1628.4, 1: 1650.6. Samples: 13086328. Policy #0 lag: (min: 28.0, avg: 28.2, max: 37.0) [2023-10-12 21:02:21,443][43579] Avg episode reward: [(0, '268.330'), (1, '257.780')] [2023-10-12 21:02:22,752][44958] Updated weights for policy 0, policy_version 25480 (0.0008) [2023-10-12 21:02:23,126][44958] Updated weights for policy 0, policy_version 25490 (0.0008) [2023-10-12 21:02:23,494][44958] Updated weights for policy 0, policy_version 25500 (0.0008) [2023-10-12 21:02:24,585][44959] Updated weights for policy 1, policy_version 25640 (0.0010) [2023-10-12 21:02:24,946][44959] Updated weights for policy 1, policy_version 25650 (0.0011) [2023-10-12 21:02:25,319][44959] Updated weights for policy 1, policy_version 25660 (0.0011) [2023-10-12 21:02:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52396032. Throughput: 0: 1624.9, 1: 1641.6. Samples: 13105572. Policy #0 lag: (min: 28.0, avg: 28.2, max: 37.0) [2023-10-12 21:02:26,444][43579] Avg episode reward: [(0, '267.420'), (1, '261.920')] [2023-10-12 21:02:27,683][44958] Updated weights for policy 0, policy_version 25510 (0.0009) [2023-10-12 21:02:28,053][44958] Updated weights for policy 0, policy_version 25520 (0.0008) [2023-10-12 21:02:28,425][44958] Updated weights for policy 0, policy_version 25530 (0.0007) [2023-10-12 21:02:29,559][44959] Updated weights for policy 1, policy_version 25670 (0.0008) [2023-10-12 21:02:29,924][44959] Updated weights for policy 1, policy_version 25680 (0.0009) [2023-10-12 21:02:30,311][44959] Updated weights for policy 1, policy_version 25690 (0.0008) [2023-10-12 21:02:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52461568. Throughput: 0: 1623.8, 1: 1645.6. Samples: 13125354. Policy #0 lag: (min: 28.0, avg: 28.2, max: 37.0) [2023-10-12 21:02:31,444][43579] Avg episode reward: [(0, '269.740'), (1, '258.350')] [2023-10-12 21:02:32,485][44958] Updated weights for policy 0, policy_version 25540 (0.0008) [2023-10-12 21:02:32,867][44958] Updated weights for policy 0, policy_version 25550 (0.0010) [2023-10-12 21:02:33,232][44958] Updated weights for policy 0, policy_version 25560 (0.0009) [2023-10-12 21:02:34,435][44959] Updated weights for policy 1, policy_version 25700 (0.0009) [2023-10-12 21:02:34,802][44959] Updated weights for policy 1, policy_version 25710 (0.0008) [2023-10-12 21:02:35,170][44959] Updated weights for policy 1, policy_version 25720 (0.0008) [2023-10-12 21:02:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52527104. Throughput: 0: 1628.2, 1: 1646.5. Samples: 13135456. Policy #0 lag: (min: 28.0, avg: 28.2, max: 37.0) [2023-10-12 21:02:36,443][43579] Avg episode reward: [(0, '271.330'), (1, '263.670')] [2023-10-12 21:02:37,473][44958] Updated weights for policy 0, policy_version 25570 (0.0010) [2023-10-12 21:02:37,837][44958] Updated weights for policy 0, policy_version 25580 (0.0008) [2023-10-12 21:02:38,212][44958] Updated weights for policy 0, policy_version 25590 (0.0008) [2023-10-12 21:02:38,577][44958] Updated weights for policy 0, policy_version 25600 (0.0009) [2023-10-12 21:02:39,228][44959] Updated weights for policy 1, policy_version 25730 (0.0007) [2023-10-12 21:02:39,599][44959] Updated weights for policy 1, policy_version 25740 (0.0009) [2023-10-12 21:02:39,970][44959] Updated weights for policy 1, policy_version 25750 (0.0007) [2023-10-12 21:02:40,333][44959] Updated weights for policy 1, policy_version 25760 (0.0008) [2023-10-12 21:02:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52592640. Throughput: 0: 1630.5, 1: 1646.8. Samples: 13155002. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-12 21:02:41,443][43579] Avg episode reward: [(0, '272.420'), (1, '269.470')] [2023-10-12 21:02:42,788][44958] Updated weights for policy 0, policy_version 25610 (0.0007) [2023-10-12 21:02:43,148][44958] Updated weights for policy 0, policy_version 25620 (0.0008) [2023-10-12 21:02:43,529][44958] Updated weights for policy 0, policy_version 25630 (0.0010) [2023-10-12 21:02:44,484][44959] Updated weights for policy 1, policy_version 25770 (0.0011) [2023-10-12 21:02:44,853][44959] Updated weights for policy 1, policy_version 25780 (0.0011) [2023-10-12 21:02:45,225][44959] Updated weights for policy 1, policy_version 25790 (0.0009) [2023-10-12 21:02:46,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52658176. Throughput: 0: 1640.0, 1: 1645.1. Samples: 13175000. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-12 21:02:46,444][43579] Avg episode reward: [(0, '277.840'), (1, '271.920')] [2023-10-12 21:02:47,474][44958] Updated weights for policy 0, policy_version 25640 (0.0009) [2023-10-12 21:02:47,846][44958] Updated weights for policy 0, policy_version 25650 (0.0007) [2023-10-12 21:02:48,229][44958] Updated weights for policy 0, policy_version 25660 (0.0009) [2023-10-12 21:02:49,424][44959] Updated weights for policy 1, policy_version 25800 (0.0008) [2023-10-12 21:02:49,787][44959] Updated weights for policy 1, policy_version 25810 (0.0008) [2023-10-12 21:02:50,158][44959] Updated weights for policy 1, policy_version 25820 (0.0009) [2023-10-12 21:02:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 52723712. Throughput: 0: 1641.1, 1: 1648.0. Samples: 13184994. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-12 21:02:51,443][43579] Avg episode reward: [(0, '276.610'), (1, '272.690')] [2023-10-12 21:02:52,551][44958] Updated weights for policy 0, policy_version 25670 (0.0008) [2023-10-12 21:02:52,930][44958] Updated weights for policy 0, policy_version 25680 (0.0007) [2023-10-12 21:02:53,295][44958] Updated weights for policy 0, policy_version 25690 (0.0007) [2023-10-12 21:02:54,364][44959] Updated weights for policy 1, policy_version 25830 (0.0009) [2023-10-12 21:02:54,735][44959] Updated weights for policy 1, policy_version 25840 (0.0010) [2023-10-12 21:02:55,105][44959] Updated weights for policy 1, policy_version 25850 (0.0010) [2023-10-12 21:02:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 52789248. Throughput: 0: 1641.9, 1: 1644.3. Samples: 13204420. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-12 21:02:56,444][43579] Avg episode reward: [(0, '279.030'), (1, '273.440')] [2023-10-12 21:02:57,531][44958] Updated weights for policy 0, policy_version 25700 (0.0008) [2023-10-12 21:02:57,910][44958] Updated weights for policy 0, policy_version 25710 (0.0008) [2023-10-12 21:02:58,289][44958] Updated weights for policy 0, policy_version 25720 (0.0008) [2023-10-12 21:02:59,220][44959] Updated weights for policy 1, policy_version 25860 (0.0009) [2023-10-12 21:02:59,583][44959] Updated weights for policy 1, policy_version 25870 (0.0010) [2023-10-12 21:02:59,950][44959] Updated weights for policy 1, policy_version 25880 (0.0010) [2023-10-12 21:03:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52854784. Throughput: 0: 1644.0, 1: 1647.2. Samples: 13224440. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-12 21:03:01,444][43579] Avg episode reward: [(0, '280.230'), (1, '278.600')] [2023-10-12 21:03:02,425][44958] Updated weights for policy 0, policy_version 25730 (0.0011) [2023-10-12 21:03:02,833][44958] Updated weights for policy 0, policy_version 25740 (0.0009) [2023-10-12 21:03:03,218][44958] Updated weights for policy 0, policy_version 25750 (0.0010) [2023-10-12 21:03:03,589][44958] Updated weights for policy 0, policy_version 25760 (0.0009) [2023-10-12 21:03:04,101][44959] Updated weights for policy 1, policy_version 25890 (0.0010) [2023-10-12 21:03:04,468][44959] Updated weights for policy 1, policy_version 25900 (0.0007) [2023-10-12 21:03:04,838][44959] Updated weights for policy 1, policy_version 25910 (0.0009) [2023-10-12 21:03:05,203][44959] Updated weights for policy 1, policy_version 25920 (0.0009) [2023-10-12 21:03:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52920320. Throughput: 0: 1644.8, 1: 1645.6. Samples: 13234398. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-12 21:03:06,444][43579] Avg episode reward: [(0, '280.760'), (1, '271.120')] [2023-10-12 21:03:07,788][44958] Updated weights for policy 0, policy_version 25770 (0.0008) [2023-10-12 21:03:08,158][44958] Updated weights for policy 0, policy_version 25780 (0.0011) [2023-10-12 21:03:08,537][44958] Updated weights for policy 0, policy_version 25790 (0.0009) [2023-10-12 21:03:09,231][44959] Updated weights for policy 1, policy_version 25930 (0.0007) [2023-10-12 21:03:09,599][44959] Updated weights for policy 1, policy_version 25940 (0.0010) [2023-10-12 21:03:09,967][44959] Updated weights for policy 1, policy_version 25950 (0.0009) [2023-10-12 21:03:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52985856. Throughput: 0: 1642.3, 1: 1642.8. Samples: 13253404. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-12 21:03:11,444][43579] Avg episode reward: [(0, '284.260'), (1, '273.030')] [2023-10-12 21:03:11,445][44518] Saving new best policy, reward=284.260! [2023-10-12 21:03:12,719][44958] Updated weights for policy 0, policy_version 25800 (0.0008) [2023-10-12 21:03:13,091][44958] Updated weights for policy 0, policy_version 25810 (0.0007) [2023-10-12 21:03:13,465][44958] Updated weights for policy 0, policy_version 25820 (0.0007) [2023-10-12 21:03:14,175][44959] Updated weights for policy 1, policy_version 25960 (0.0010) [2023-10-12 21:03:14,549][44959] Updated weights for policy 1, policy_version 25970 (0.0008) [2023-10-12 21:03:14,917][44959] Updated weights for policy 1, policy_version 25980 (0.0009) [2023-10-12 21:03:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 53051392. Throughput: 0: 1650.1, 1: 1656.5. Samples: 13274150. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-12 21:03:16,443][43579] Avg episode reward: [(0, '281.350'), (1, '275.750')] [2023-10-12 21:03:17,361][44958] Updated weights for policy 0, policy_version 25830 (0.0008) [2023-10-12 21:03:17,738][44958] Updated weights for policy 0, policy_version 25840 (0.0007) [2023-10-12 21:03:18,108][44958] Updated weights for policy 0, policy_version 25850 (0.0007) [2023-10-12 21:03:19,105][44959] Updated weights for policy 1, policy_version 25990 (0.0008) [2023-10-12 21:03:19,477][44959] Updated weights for policy 1, policy_version 26000 (0.0008) [2023-10-12 21:03:19,834][44959] Updated weights for policy 1, policy_version 26010 (0.0009) [2023-10-12 21:03:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53116928. Throughput: 0: 1651.2, 1: 1654.5. Samples: 13284212. Policy #0 lag: (min: 5.0, avg: 7.3, max: 37.0) [2023-10-12 21:03:21,444][43579] Avg episode reward: [(0, '277.880'), (1, '273.720')] [2023-10-12 21:03:22,388][44958] Updated weights for policy 0, policy_version 25860 (0.0009) [2023-10-12 21:03:22,769][44958] Updated weights for policy 0, policy_version 25870 (0.0008) [2023-10-12 21:03:23,140][44958] Updated weights for policy 0, policy_version 25880 (0.0007) [2023-10-12 21:03:23,921][44959] Updated weights for policy 1, policy_version 26020 (0.0008) [2023-10-12 21:03:24,292][44959] Updated weights for policy 1, policy_version 26030 (0.0008) [2023-10-12 21:03:24,649][44959] Updated weights for policy 1, policy_version 26040 (0.0011) [2023-10-12 21:03:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53182464. Throughput: 0: 1643.2, 1: 1645.7. Samples: 13303000. Policy #0 lag: (min: 5.0, avg: 7.3, max: 37.0) [2023-10-12 21:03:26,443][43579] Avg episode reward: [(0, '277.610'), (1, '277.640')] [2023-10-12 21:03:27,391][44958] Updated weights for policy 0, policy_version 25890 (0.0008) [2023-10-12 21:03:27,771][44958] Updated weights for policy 0, policy_version 25900 (0.0009) [2023-10-12 21:03:28,145][44958] Updated weights for policy 0, policy_version 25910 (0.0008) [2023-10-12 21:03:28,524][44958] Updated weights for policy 0, policy_version 25920 (0.0009) [2023-10-12 21:03:28,991][44959] Updated weights for policy 1, policy_version 26050 (0.0008) [2023-10-12 21:03:29,405][44959] Updated weights for policy 1, policy_version 26060 (0.0010) [2023-10-12 21:03:29,781][44959] Updated weights for policy 1, policy_version 26070 (0.0009) [2023-10-12 21:03:30,149][44959] Updated weights for policy 1, policy_version 26080 (0.0009) [2023-10-12 21:03:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 53248000. Throughput: 0: 1641.7, 1: 1653.5. Samples: 13323284. Policy #0 lag: (min: 5.0, avg: 7.3, max: 37.0) [2023-10-12 21:03:31,443][43579] Avg episode reward: [(0, '275.310'), (1, '270.890')] [2023-10-12 21:03:32,349][44958] Updated weights for policy 0, policy_version 25930 (0.0008) [2023-10-12 21:03:32,720][44958] Updated weights for policy 0, policy_version 25940 (0.0007) [2023-10-12 21:03:33,089][44958] Updated weights for policy 0, policy_version 25950 (0.0009) [2023-10-12 21:03:34,084][44959] Updated weights for policy 1, policy_version 26090 (0.0007) [2023-10-12 21:03:34,444][44959] Updated weights for policy 1, policy_version 26100 (0.0009) [2023-10-12 21:03:34,821][44959] Updated weights for policy 1, policy_version 26110 (0.0008) [2023-10-12 21:03:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53313536. Throughput: 0: 1640.4, 1: 1654.4. Samples: 13333258. Policy #0 lag: (min: 5.0, avg: 7.3, max: 37.0) [2023-10-12 21:03:36,443][43579] Avg episode reward: [(0, '271.810'), (1, '273.790')] [2023-10-12 21:03:37,511][44958] Updated weights for policy 0, policy_version 25960 (0.0008) [2023-10-12 21:03:37,889][44958] Updated weights for policy 0, policy_version 25970 (0.0008) [2023-10-12 21:03:38,258][44958] Updated weights for policy 0, policy_version 25980 (0.0007) [2023-10-12 21:03:39,038][44959] Updated weights for policy 1, policy_version 26120 (0.0008) [2023-10-12 21:03:39,400][44959] Updated weights for policy 1, policy_version 26130 (0.0010) [2023-10-12 21:03:39,772][44959] Updated weights for policy 1, policy_version 26140 (0.0007) [2023-10-12 21:03:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53379072. Throughput: 0: 1639.4, 1: 1647.7. Samples: 13352342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:03:41,443][43579] Avg episode reward: [(0, '273.690'), (1, '268.690')] [2023-10-12 21:03:42,376][44958] Updated weights for policy 0, policy_version 25990 (0.0008) [2023-10-12 21:03:42,743][44958] Updated weights for policy 0, policy_version 26000 (0.0008) [2023-10-12 21:03:43,123][44958] Updated weights for policy 0, policy_version 26010 (0.0009) [2023-10-12 21:03:43,875][44959] Updated weights for policy 1, policy_version 26150 (0.0007) [2023-10-12 21:03:44,235][44959] Updated weights for policy 1, policy_version 26160 (0.0008) [2023-10-12 21:03:44,613][44959] Updated weights for policy 1, policy_version 26170 (0.0007) [2023-10-12 21:03:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53444608. Throughput: 0: 1643.2, 1: 1656.0. Samples: 13372904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:03:46,444][43579] Avg episode reward: [(0, '271.140'), (1, '269.260')] [2023-10-12 21:03:46,458][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000026176_26804224.pth... [2023-10-12 21:03:46,458][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000026016_26640384.pth... [2023-10-12 21:03:46,494][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000024640_25231360.pth [2023-10-12 21:03:46,494][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000024480_25067520.pth [2023-10-12 21:03:47,203][44958] Updated weights for policy 0, policy_version 26020 (0.0008) [2023-10-12 21:03:47,589][44958] Updated weights for policy 0, policy_version 26030 (0.0009) [2023-10-12 21:03:47,958][44958] Updated weights for policy 0, policy_version 26040 (0.0010) [2023-10-12 21:03:48,753][44959] Updated weights for policy 1, policy_version 26180 (0.0007) [2023-10-12 21:03:49,120][44959] Updated weights for policy 1, policy_version 26190 (0.0008) [2023-10-12 21:03:49,482][44959] Updated weights for policy 1, policy_version 26200 (0.0010) [2023-10-12 21:03:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53510144. Throughput: 0: 1641.3, 1: 1646.5. Samples: 13382350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:03:51,444][43579] Avg episode reward: [(0, '270.770'), (1, '261.170')] [2023-10-12 21:03:52,264][44958] Updated weights for policy 0, policy_version 26050 (0.0011) [2023-10-12 21:03:52,632][44958] Updated weights for policy 0, policy_version 26060 (0.0008) [2023-10-12 21:03:53,010][44958] Updated weights for policy 0, policy_version 26070 (0.0007) [2023-10-12 21:03:53,382][44958] Updated weights for policy 0, policy_version 26080 (0.0008) [2023-10-12 21:03:53,644][44959] Updated weights for policy 1, policy_version 26210 (0.0009) [2023-10-12 21:03:54,005][44959] Updated weights for policy 1, policy_version 26220 (0.0009) [2023-10-12 21:03:54,376][44959] Updated weights for policy 1, policy_version 26230 (0.0009) [2023-10-12 21:03:54,743][44959] Updated weights for policy 1, policy_version 26240 (0.0008) [2023-10-12 21:03:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53575680. Throughput: 0: 1647.1, 1: 1650.7. Samples: 13401806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:03:56,444][43579] Avg episode reward: [(0, '270.290'), (1, '260.140')] [2023-10-12 21:03:57,524][44958] Updated weights for policy 0, policy_version 26090 (0.0010) [2023-10-12 21:03:57,897][44958] Updated weights for policy 0, policy_version 26100 (0.0008) [2023-10-12 21:03:58,275][44958] Updated weights for policy 0, policy_version 26110 (0.0008) [2023-10-12 21:03:58,935][44959] Updated weights for policy 1, policy_version 26250 (0.0010) [2023-10-12 21:03:59,307][44959] Updated weights for policy 1, policy_version 26260 (0.0009) [2023-10-12 21:03:59,676][44959] Updated weights for policy 1, policy_version 26270 (0.0008) [2023-10-12 21:04:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53641216. Throughput: 0: 1638.9, 1: 1648.0. Samples: 13422062. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-12 21:04:01,444][43579] Avg episode reward: [(0, '268.250'), (1, '262.450')] [2023-10-12 21:04:02,488][44958] Updated weights for policy 0, policy_version 26120 (0.0008) [2023-10-12 21:04:02,856][44958] Updated weights for policy 0, policy_version 26130 (0.0008) [2023-10-12 21:04:03,234][44958] Updated weights for policy 0, policy_version 26140 (0.0008) [2023-10-12 21:04:04,050][44959] Updated weights for policy 1, policy_version 26280 (0.0009) [2023-10-12 21:04:04,420][44959] Updated weights for policy 1, policy_version 26290 (0.0010) [2023-10-12 21:04:04,790][44959] Updated weights for policy 1, policy_version 26300 (0.0010) [2023-10-12 21:04:06,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53706752. Throughput: 0: 1633.4, 1: 1641.1. Samples: 13431564. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-12 21:04:06,443][43579] Avg episode reward: [(0, '269.990'), (1, '264.500')] [2023-10-12 21:04:07,465][44958] Updated weights for policy 0, policy_version 26150 (0.0008) [2023-10-12 21:04:07,838][44958] Updated weights for policy 0, policy_version 26160 (0.0007) [2023-10-12 21:04:08,222][44958] Updated weights for policy 0, policy_version 26170 (0.0008) [2023-10-12 21:04:08,902][44959] Updated weights for policy 1, policy_version 26310 (0.0009) [2023-10-12 21:04:09,276][44959] Updated weights for policy 1, policy_version 26320 (0.0007) [2023-10-12 21:04:09,645][44959] Updated weights for policy 1, policy_version 26330 (0.0007) [2023-10-12 21:04:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53772288. Throughput: 0: 1637.3, 1: 1642.4. Samples: 13450586. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-12 21:04:11,443][43579] Avg episode reward: [(0, '270.740'), (1, '260.260')] [2023-10-12 21:04:12,364][44958] Updated weights for policy 0, policy_version 26180 (0.0008) [2023-10-12 21:04:12,742][44958] Updated weights for policy 0, policy_version 26190 (0.0008) [2023-10-12 21:04:13,109][44958] Updated weights for policy 0, policy_version 26200 (0.0008) [2023-10-12 21:04:13,815][44959] Updated weights for policy 1, policy_version 26340 (0.0009) [2023-10-12 21:04:14,200][44959] Updated weights for policy 1, policy_version 26350 (0.0009) [2023-10-12 21:04:14,569][44959] Updated weights for policy 1, policy_version 26360 (0.0007) [2023-10-12 21:04:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53837824. Throughput: 0: 1627.8, 1: 1650.5. Samples: 13470808. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-12 21:04:16,443][43579] Avg episode reward: [(0, '274.800'), (1, '262.210')] [2023-10-12 21:04:17,481][44958] Updated weights for policy 0, policy_version 26210 (0.0010) [2023-10-12 21:04:17,848][44958] Updated weights for policy 0, policy_version 26220 (0.0008) [2023-10-12 21:04:18,217][44958] Updated weights for policy 0, policy_version 26230 (0.0007) [2023-10-12 21:04:18,593][44958] Updated weights for policy 0, policy_version 26240 (0.0009) [2023-10-12 21:04:18,799][44959] Updated weights for policy 1, policy_version 26370 (0.0009) [2023-10-12 21:04:19,162][44959] Updated weights for policy 1, policy_version 26380 (0.0010) [2023-10-12 21:04:19,539][44959] Updated weights for policy 1, policy_version 26390 (0.0009) [2023-10-12 21:04:19,902][44959] Updated weights for policy 1, policy_version 26400 (0.0009) [2023-10-12 21:04:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53903360. Throughput: 0: 1627.3, 1: 1643.0. Samples: 13480422. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 21:04:21,444][43579] Avg episode reward: [(0, '277.530'), (1, '268.050')] [2023-10-12 21:04:22,580][44958] Updated weights for policy 0, policy_version 26250 (0.0007) [2023-10-12 21:04:22,964][44958] Updated weights for policy 0, policy_version 26260 (0.0008) [2023-10-12 21:04:23,340][44958] Updated weights for policy 0, policy_version 26270 (0.0009) [2023-10-12 21:04:24,129][44959] Updated weights for policy 1, policy_version 26410 (0.0009) [2023-10-12 21:04:24,497][44959] Updated weights for policy 1, policy_version 26420 (0.0008) [2023-10-12 21:04:24,867][44959] Updated weights for policy 1, policy_version 26430 (0.0008) [2023-10-12 21:04:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53968896. Throughput: 0: 1633.8, 1: 1644.6. Samples: 13499870. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 21:04:26,444][43579] Avg episode reward: [(0, '279.930'), (1, '270.580')] [2023-10-12 21:04:27,514][44958] Updated weights for policy 0, policy_version 26280 (0.0008) [2023-10-12 21:04:27,874][44958] Updated weights for policy 0, policy_version 26290 (0.0009) [2023-10-12 21:04:28,252][44958] Updated weights for policy 0, policy_version 26300 (0.0008) [2023-10-12 21:04:28,867][44959] Updated weights for policy 1, policy_version 26440 (0.0008) [2023-10-12 21:04:29,240][44959] Updated weights for policy 1, policy_version 26450 (0.0009) [2023-10-12 21:04:29,620][44959] Updated weights for policy 1, policy_version 26460 (0.0008) [2023-10-12 21:04:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 54034432. Throughput: 0: 1628.8, 1: 1648.6. Samples: 13520388. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 21:04:31,444][43579] Avg episode reward: [(0, '283.160'), (1, '274.000')] [2023-10-12 21:04:32,668][44958] Updated weights for policy 0, policy_version 26310 (0.0008) [2023-10-12 21:04:33,051][44958] Updated weights for policy 0, policy_version 26320 (0.0008) [2023-10-12 21:04:33,419][44958] Updated weights for policy 0, policy_version 26330 (0.0010) [2023-10-12 21:04:33,906][44959] Updated weights for policy 1, policy_version 26470 (0.0010) [2023-10-12 21:04:34,271][44959] Updated weights for policy 1, policy_version 26480 (0.0009) [2023-10-12 21:04:34,646][44959] Updated weights for policy 1, policy_version 26490 (0.0011) [2023-10-12 21:04:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54099968. Throughput: 0: 1629.9, 1: 1650.7. Samples: 13529976. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 21:04:36,443][43579] Avg episode reward: [(0, '283.910'), (1, '268.540')] [2023-10-12 21:04:37,504][44958] Updated weights for policy 0, policy_version 26340 (0.0010) [2023-10-12 21:04:37,873][44958] Updated weights for policy 0, policy_version 26350 (0.0007) [2023-10-12 21:04:38,249][44958] Updated weights for policy 0, policy_version 26360 (0.0008) [2023-10-12 21:04:38,669][44959] Updated weights for policy 1, policy_version 26500 (0.0010) [2023-10-12 21:04:39,044][44959] Updated weights for policy 1, policy_version 26510 (0.0009) [2023-10-12 21:04:39,415][44959] Updated weights for policy 1, policy_version 26520 (0.0007) [2023-10-12 21:04:41,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54165504. Throughput: 0: 1629.3, 1: 1653.8. Samples: 13549544. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-12 21:04:41,443][43579] Avg episode reward: [(0, '284.090'), (1, '274.790')] [2023-10-12 21:04:42,617][44958] Updated weights for policy 0, policy_version 26370 (0.0008) [2023-10-12 21:04:42,981][44958] Updated weights for policy 0, policy_version 26380 (0.0008) [2023-10-12 21:04:43,360][44958] Updated weights for policy 0, policy_version 26390 (0.0007) [2023-10-12 21:04:43,380][44959] Updated weights for policy 1, policy_version 26530 (0.0008) [2023-10-12 21:04:43,720][44958] Updated weights for policy 0, policy_version 26400 (0.0009) [2023-10-12 21:04:43,753][44959] Updated weights for policy 1, policy_version 26540 (0.0007) [2023-10-12 21:04:44,120][44959] Updated weights for policy 1, policy_version 26550 (0.0007) [2023-10-12 21:04:44,483][44959] Updated weights for policy 1, policy_version 26560 (0.0008) [2023-10-12 21:04:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54231040. Throughput: 0: 1631.2, 1: 1654.6. Samples: 13569922. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-12 21:04:46,444][43579] Avg episode reward: [(0, '283.850'), (1, '275.260')] [2023-10-12 21:04:47,857][44958] Updated weights for policy 0, policy_version 26410 (0.0009) [2023-10-12 21:04:48,230][44958] Updated weights for policy 0, policy_version 26420 (0.0008) [2023-10-12 21:04:48,604][44958] Updated weights for policy 0, policy_version 26430 (0.0007) [2023-10-12 21:04:48,724][44959] Updated weights for policy 1, policy_version 26570 (0.0009) [2023-10-12 21:04:49,093][44959] Updated weights for policy 1, policy_version 26580 (0.0009) [2023-10-12 21:04:49,470][44959] Updated weights for policy 1, policy_version 26590 (0.0008) [2023-10-12 21:04:51,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54296576. Throughput: 0: 1633.8, 1: 1647.5. Samples: 13579224. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-12 21:04:51,444][43579] Avg episode reward: [(0, '276.970'), (1, '270.060')] [2023-10-12 21:04:52,624][44958] Updated weights for policy 0, policy_version 26440 (0.0008) [2023-10-12 21:04:52,999][44958] Updated weights for policy 0, policy_version 26450 (0.0007) [2023-10-12 21:04:53,380][44958] Updated weights for policy 0, policy_version 26460 (0.0009) [2023-10-12 21:04:53,725][44959] Updated weights for policy 1, policy_version 26600 (0.0011) [2023-10-12 21:04:54,096][44959] Updated weights for policy 1, policy_version 26610 (0.0010) [2023-10-12 21:04:54,468][44959] Updated weights for policy 1, policy_version 26620 (0.0008) [2023-10-12 21:04:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54362112. Throughput: 0: 1642.5, 1: 1660.3. Samples: 13599212. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-12 21:04:56,444][43579] Avg episode reward: [(0, '272.100'), (1, '267.210')] [2023-10-12 21:04:57,609][44958] Updated weights for policy 0, policy_version 26470 (0.0008) [2023-10-12 21:04:57,986][44958] Updated weights for policy 0, policy_version 26480 (0.0009) [2023-10-12 21:04:58,373][44958] Updated weights for policy 0, policy_version 26490 (0.0010) [2023-10-12 21:04:58,687][44959] Updated weights for policy 1, policy_version 26630 (0.0008) [2023-10-12 21:04:59,047][44959] Updated weights for policy 1, policy_version 26640 (0.0011) [2023-10-12 21:04:59,422][44959] Updated weights for policy 1, policy_version 26650 (0.0009) [2023-10-12 21:05:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54427648. Throughput: 0: 1645.4, 1: 1652.2. Samples: 13619200. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:05:01,444][43579] Avg episode reward: [(0, '272.260'), (1, '266.290')] [2023-10-12 21:05:02,489][44958] Updated weights for policy 0, policy_version 26500 (0.0009) [2023-10-12 21:05:02,869][44958] Updated weights for policy 0, policy_version 26510 (0.0009) [2023-10-12 21:05:03,246][44958] Updated weights for policy 0, policy_version 26520 (0.0010) [2023-10-12 21:05:03,631][44959] Updated weights for policy 1, policy_version 26660 (0.0009) [2023-10-12 21:05:04,041][44959] Updated weights for policy 1, policy_version 26670 (0.0008) [2023-10-12 21:05:04,403][44959] Updated weights for policy 1, policy_version 26680 (0.0008) [2023-10-12 21:05:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54493184. Throughput: 0: 1649.3, 1: 1645.3. Samples: 13628680. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:05:06,443][43579] Avg episode reward: [(0, '266.290'), (1, '272.520')] [2023-10-12 21:05:07,278][44958] Updated weights for policy 0, policy_version 26530 (0.0008) [2023-10-12 21:05:07,654][44958] Updated weights for policy 0, policy_version 26540 (0.0007) [2023-10-12 21:05:08,025][44958] Updated weights for policy 0, policy_version 26550 (0.0008) [2023-10-12 21:05:08,391][44958] Updated weights for policy 0, policy_version 26560 (0.0009) [2023-10-12 21:05:08,546][44959] Updated weights for policy 1, policy_version 26690 (0.0008) [2023-10-12 21:05:08,920][44959] Updated weights for policy 1, policy_version 26700 (0.0009) [2023-10-12 21:05:09,295][44959] Updated weights for policy 1, policy_version 26710 (0.0011) [2023-10-12 21:05:09,660][44959] Updated weights for policy 1, policy_version 26720 (0.0009) [2023-10-12 21:05:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54558720. Throughput: 0: 1649.8, 1: 1647.2. Samples: 13648236. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:05:11,443][43579] Avg episode reward: [(0, '266.770'), (1, '268.730')] [2023-10-12 21:05:12,515][44958] Updated weights for policy 0, policy_version 26570 (0.0007) [2023-10-12 21:05:12,889][44958] Updated weights for policy 0, policy_version 26580 (0.0009) [2023-10-12 21:05:13,275][44958] Updated weights for policy 0, policy_version 26590 (0.0008) [2023-10-12 21:05:13,667][44959] Updated weights for policy 1, policy_version 26730 (0.0008) [2023-10-12 21:05:14,029][44959] Updated weights for policy 1, policy_version 26740 (0.0012) [2023-10-12 21:05:14,406][44959] Updated weights for policy 1, policy_version 26750 (0.0010) [2023-10-12 21:05:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54624256. Throughput: 0: 1654.4, 1: 1647.0. Samples: 13668948. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:05:16,444][43579] Avg episode reward: [(0, '269.990'), (1, '269.410')] [2023-10-12 21:05:17,295][44958] Updated weights for policy 0, policy_version 26600 (0.0008) [2023-10-12 21:05:17,677][44958] Updated weights for policy 0, policy_version 26610 (0.0011) [2023-10-12 21:05:18,050][44958] Updated weights for policy 0, policy_version 26620 (0.0009) [2023-10-12 21:05:18,444][44959] Updated weights for policy 1, policy_version 26760 (0.0010) [2023-10-12 21:05:18,820][44959] Updated weights for policy 1, policy_version 26770 (0.0008) [2023-10-12 21:05:19,181][44959] Updated weights for policy 1, policy_version 26780 (0.0008) [2023-10-12 21:05:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54689792. Throughput: 0: 1655.1, 1: 1634.3. Samples: 13678000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:05:21,443][43579] Avg episode reward: [(0, '271.330'), (1, '272.820')] [2023-10-12 21:05:22,132][44958] Updated weights for policy 0, policy_version 26630 (0.0007) [2023-10-12 21:05:22,503][44958] Updated weights for policy 0, policy_version 26640 (0.0007) [2023-10-12 21:05:22,876][44958] Updated weights for policy 0, policy_version 26650 (0.0007) [2023-10-12 21:05:23,352][44959] Updated weights for policy 1, policy_version 26790 (0.0008) [2023-10-12 21:05:23,707][44959] Updated weights for policy 1, policy_version 26800 (0.0010) [2023-10-12 21:05:24,074][44959] Updated weights for policy 1, policy_version 26810 (0.0008) [2023-10-12 21:05:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54755328. Throughput: 0: 1651.4, 1: 1642.2. Samples: 13697756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:05:26,443][43579] Avg episode reward: [(0, '274.850'), (1, '272.240')] [2023-10-12 21:05:27,163][44958] Updated weights for policy 0, policy_version 26660 (0.0007) [2023-10-12 21:05:27,542][44958] Updated weights for policy 0, policy_version 26670 (0.0008) [2023-10-12 21:05:27,901][44958] Updated weights for policy 0, policy_version 26680 (0.0008) [2023-10-12 21:05:28,225][44959] Updated weights for policy 1, policy_version 26820 (0.0007) [2023-10-12 21:05:28,603][44959] Updated weights for policy 1, policy_version 26830 (0.0007) [2023-10-12 21:05:28,973][44959] Updated weights for policy 1, policy_version 26840 (0.0010) [2023-10-12 21:05:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54820864. Throughput: 0: 1647.5, 1: 1640.1. Samples: 13717864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:05:31,444][43579] Avg episode reward: [(0, '274.350'), (1, '273.600')] [2023-10-12 21:05:32,050][44958] Updated weights for policy 0, policy_version 26690 (0.0008) [2023-10-12 21:05:32,415][44958] Updated weights for policy 0, policy_version 26700 (0.0009) [2023-10-12 21:05:32,789][44958] Updated weights for policy 0, policy_version 26710 (0.0007) [2023-10-12 21:05:33,126][44959] Updated weights for policy 1, policy_version 26850 (0.0009) [2023-10-12 21:05:33,165][44958] Updated weights for policy 0, policy_version 26720 (0.0007) [2023-10-12 21:05:33,493][44959] Updated weights for policy 1, policy_version 26860 (0.0009) [2023-10-12 21:05:33,850][44959] Updated weights for policy 1, policy_version 26870 (0.0009) [2023-10-12 21:05:34,222][44959] Updated weights for policy 1, policy_version 26880 (0.0009) [2023-10-12 21:05:36,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 54886400. Throughput: 0: 1649.1, 1: 1636.5. Samples: 13727074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:05:36,444][43579] Avg episode reward: [(0, '276.740'), (1, '271.990')] [2023-10-12 21:05:37,368][44958] Updated weights for policy 0, policy_version 26730 (0.0008) [2023-10-12 21:05:37,747][44958] Updated weights for policy 0, policy_version 26740 (0.0010) [2023-10-12 21:05:38,117][44958] Updated weights for policy 0, policy_version 26750 (0.0008) [2023-10-12 21:05:38,225][44959] Updated weights for policy 1, policy_version 26890 (0.0007) [2023-10-12 21:05:38,590][44959] Updated weights for policy 1, policy_version 26900 (0.0010) [2023-10-12 21:05:38,968][44959] Updated weights for policy 1, policy_version 26910 (0.0009) [2023-10-12 21:05:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 54951936. Throughput: 0: 1643.9, 1: 1646.0. Samples: 13747254. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-12 21:05:41,443][43579] Avg episode reward: [(0, '274.510'), (1, '275.430')] [2023-10-12 21:05:42,288][44958] Updated weights for policy 0, policy_version 26760 (0.0007) [2023-10-12 21:05:42,664][44958] Updated weights for policy 0, policy_version 26770 (0.0007) [2023-10-12 21:05:43,043][44958] Updated weights for policy 0, policy_version 26780 (0.0007) [2023-10-12 21:05:43,381][44959] Updated weights for policy 1, policy_version 26920 (0.0010) [2023-10-12 21:05:43,753][44959] Updated weights for policy 1, policy_version 26930 (0.0009) [2023-10-12 21:05:44,126][44959] Updated weights for policy 1, policy_version 26940 (0.0008) [2023-10-12 21:05:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55017472. Throughput: 0: 1646.8, 1: 1647.4. Samples: 13767438. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-12 21:05:46,444][43579] Avg episode reward: [(0, '274.730'), (1, '271.310')] [2023-10-12 21:05:46,457][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000026944_27590656.pth... [2023-10-12 21:05:46,457][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000026784_27426816.pth... [2023-10-12 21:05:46,494][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000025248_25853952.pth [2023-10-12 21:05:46,496][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000025408_26017792.pth [2023-10-12 21:05:47,184][44958] Updated weights for policy 0, policy_version 26790 (0.0007) [2023-10-12 21:05:47,556][44958] Updated weights for policy 0, policy_version 26800 (0.0007) [2023-10-12 21:05:47,932][44958] Updated weights for policy 0, policy_version 26810 (0.0008) [2023-10-12 21:05:48,476][44959] Updated weights for policy 1, policy_version 26950 (0.0007) [2023-10-12 21:05:48,868][44959] Updated weights for policy 1, policy_version 26960 (0.0008) [2023-10-12 21:05:49,237][44959] Updated weights for policy 1, policy_version 26970 (0.0007) [2023-10-12 21:05:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55083008. Throughput: 0: 1645.5, 1: 1641.2. Samples: 13776586. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-12 21:05:51,444][43579] Avg episode reward: [(0, '274.100'), (1, '273.830')] [2023-10-12 21:05:52,139][44958] Updated weights for policy 0, policy_version 26820 (0.0008) [2023-10-12 21:05:52,514][44958] Updated weights for policy 0, policy_version 26830 (0.0007) [2023-10-12 21:05:52,879][44958] Updated weights for policy 0, policy_version 26840 (0.0010) [2023-10-12 21:05:53,299][44959] Updated weights for policy 1, policy_version 26980 (0.0007) [2023-10-12 21:05:53,678][44959] Updated weights for policy 1, policy_version 26990 (0.0009) [2023-10-12 21:05:54,034][44959] Updated weights for policy 1, policy_version 27000 (0.0008) [2023-10-12 21:05:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55148544. Throughput: 0: 1644.0, 1: 1654.1. Samples: 13796650. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-12 21:05:56,444][43579] Avg episode reward: [(0, '275.440'), (1, '269.870')] [2023-10-12 21:05:56,965][44958] Updated weights for policy 0, policy_version 26850 (0.0009) [2023-10-12 21:05:57,335][44958] Updated weights for policy 0, policy_version 26860 (0.0008) [2023-10-12 21:05:57,711][44958] Updated weights for policy 0, policy_version 26870 (0.0008) [2023-10-12 21:05:58,088][44958] Updated weights for policy 0, policy_version 26880 (0.0008) [2023-10-12 21:05:58,169][44959] Updated weights for policy 1, policy_version 27010 (0.0008) [2023-10-12 21:05:58,527][44959] Updated weights for policy 1, policy_version 27020 (0.0010) [2023-10-12 21:05:58,892][44959] Updated weights for policy 1, policy_version 27030 (0.0008) [2023-10-12 21:05:59,265][44959] Updated weights for policy 1, policy_version 27040 (0.0008) [2023-10-12 21:06:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55214080. Throughput: 0: 1638.9, 1: 1645.7. Samples: 13816756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:06:01,444][43579] Avg episode reward: [(0, '271.490'), (1, '267.610')] [2023-10-12 21:06:02,303][44958] Updated weights for policy 0, policy_version 26890 (0.0007) [2023-10-12 21:06:02,676][44958] Updated weights for policy 0, policy_version 26900 (0.0007) [2023-10-12 21:06:03,051][44958] Updated weights for policy 0, policy_version 26910 (0.0007) [2023-10-12 21:06:03,470][44959] Updated weights for policy 1, policy_version 27050 (0.0010) [2023-10-12 21:06:03,834][44959] Updated weights for policy 1, policy_version 27060 (0.0010) [2023-10-12 21:06:04,203][44959] Updated weights for policy 1, policy_version 27070 (0.0009) [2023-10-12 21:06:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 55279616. Throughput: 0: 1641.5, 1: 1650.2. Samples: 13826128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:06:06,444][43579] Avg episode reward: [(0, '271.770'), (1, '260.260')] [2023-10-12 21:06:07,394][44958] Updated weights for policy 0, policy_version 26920 (0.0009) [2023-10-12 21:06:07,772][44958] Updated weights for policy 0, policy_version 26930 (0.0007) [2023-10-12 21:06:08,146][44958] Updated weights for policy 0, policy_version 26940 (0.0007) [2023-10-12 21:06:08,222][44959] Updated weights for policy 1, policy_version 27080 (0.0007) [2023-10-12 21:06:08,588][44959] Updated weights for policy 1, policy_version 27090 (0.0009) [2023-10-12 21:06:08,968][44959] Updated weights for policy 1, policy_version 27100 (0.0011) [2023-10-12 21:06:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55345152. Throughput: 0: 1635.5, 1: 1655.7. Samples: 13845860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:06:11,444][43579] Avg episode reward: [(0, '269.710'), (1, '264.860')] [2023-10-12 21:06:12,280][44958] Updated weights for policy 0, policy_version 26950 (0.0009) [2023-10-12 21:06:12,656][44958] Updated weights for policy 0, policy_version 26960 (0.0010) [2023-10-12 21:06:13,028][44958] Updated weights for policy 0, policy_version 26970 (0.0010) [2023-10-12 21:06:13,137][44959] Updated weights for policy 1, policy_version 27110 (0.0009) [2023-10-12 21:06:13,506][44959] Updated weights for policy 1, policy_version 27120 (0.0009) [2023-10-12 21:06:13,883][44959] Updated weights for policy 1, policy_version 27130 (0.0009) [2023-10-12 21:06:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55410688. Throughput: 0: 1634.3, 1: 1664.1. Samples: 13866292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:06:16,444][43579] Avg episode reward: [(0, '270.020'), (1, '259.620')] [2023-10-12 21:06:17,169][44958] Updated weights for policy 0, policy_version 26980 (0.0011) [2023-10-12 21:06:17,543][44958] Updated weights for policy 0, policy_version 26990 (0.0010) [2023-10-12 21:06:17,857][44959] Updated weights for policy 1, policy_version 27140 (0.0008) [2023-10-12 21:06:17,925][44958] Updated weights for policy 0, policy_version 27000 (0.0008) [2023-10-12 21:06:18,226][44959] Updated weights for policy 1, policy_version 27150 (0.0007) [2023-10-12 21:06:18,597][44959] Updated weights for policy 1, policy_version 27160 (0.0007) [2023-10-12 21:06:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 55476224. Throughput: 0: 1632.2, 1: 1658.7. Samples: 13875164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:06:21,444][43579] Avg episode reward: [(0, '271.570'), (1, '262.120')] [2023-10-12 21:06:22,284][44958] Updated weights for policy 0, policy_version 27010 (0.0008) [2023-10-12 21:06:22,641][44959] Updated weights for policy 1, policy_version 27170 (0.0008) [2023-10-12 21:06:22,661][44958] Updated weights for policy 0, policy_version 27020 (0.0008) [2023-10-12 21:06:22,997][44959] Updated weights for policy 1, policy_version 27180 (0.0010) [2023-10-12 21:06:23,030][44958] Updated weights for policy 0, policy_version 27030 (0.0008) [2023-10-12 21:06:23,373][44959] Updated weights for policy 1, policy_version 27190 (0.0009) [2023-10-12 21:06:23,398][44958] Updated weights for policy 0, policy_version 27040 (0.0007) [2023-10-12 21:06:23,736][44959] Updated weights for policy 1, policy_version 27200 (0.0010) [2023-10-12 21:06:26,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55541760. Throughput: 0: 1630.4, 1: 1660.2. Samples: 13895328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:06:26,444][43579] Avg episode reward: [(0, '274.070'), (1, '256.570')] [2023-10-12 21:06:27,547][44958] Updated weights for policy 0, policy_version 27050 (0.0009) [2023-10-12 21:06:27,870][44959] Updated weights for policy 1, policy_version 27210 (0.0009) [2023-10-12 21:06:27,913][44958] Updated weights for policy 0, policy_version 27060 (0.0007) [2023-10-12 21:06:28,231][44959] Updated weights for policy 1, policy_version 27220 (0.0008) [2023-10-12 21:06:28,291][44958] Updated weights for policy 0, policy_version 27070 (0.0007) [2023-10-12 21:06:28,600][44959] Updated weights for policy 1, policy_version 27230 (0.0011) [2023-10-12 21:06:31,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55607296. Throughput: 0: 1632.3, 1: 1665.2. Samples: 13915822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:06:31,444][43579] Avg episode reward: [(0, '275.130'), (1, '258.800')] [2023-10-12 21:06:32,386][44958] Updated weights for policy 0, policy_version 27080 (0.0008) [2023-10-12 21:06:32,756][44958] Updated weights for policy 0, policy_version 27090 (0.0009) [2023-10-12 21:06:32,761][44959] Updated weights for policy 1, policy_version 27240 (0.0008) [2023-10-12 21:06:33,124][44958] Updated weights for policy 0, policy_version 27100 (0.0010) [2023-10-12 21:06:33,131][44959] Updated weights for policy 1, policy_version 27250 (0.0008) [2023-10-12 21:06:33,491][44959] Updated weights for policy 1, policy_version 27260 (0.0008) [2023-10-12 21:06:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55672832. Throughput: 0: 1630.8, 1: 1656.4. Samples: 13924508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:06:36,444][43579] Avg episode reward: [(0, '275.760'), (1, '268.420')] [2023-10-12 21:06:37,381][44958] Updated weights for policy 0, policy_version 27110 (0.0007) [2023-10-12 21:06:37,614][44959] Updated weights for policy 1, policy_version 27270 (0.0009) [2023-10-12 21:06:37,754][44958] Updated weights for policy 0, policy_version 27120 (0.0007) [2023-10-12 21:06:37,979][44959] Updated weights for policy 1, policy_version 27280 (0.0009) [2023-10-12 21:06:38,133][44958] Updated weights for policy 0, policy_version 27130 (0.0008) [2023-10-12 21:06:38,343][44959] Updated weights for policy 1, policy_version 27290 (0.0007) [2023-10-12 21:06:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55738368. Throughput: 0: 1625.7, 1: 1660.4. Samples: 13944526. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) [2023-10-12 21:06:41,443][43579] Avg episode reward: [(0, '278.860'), (1, '266.110')] [2023-10-12 21:06:42,422][44958] Updated weights for policy 0, policy_version 27140 (0.0009) [2023-10-12 21:06:42,671][44959] Updated weights for policy 1, policy_version 27300 (0.0009) [2023-10-12 21:06:42,786][44958] Updated weights for policy 0, policy_version 27150 (0.0009) [2023-10-12 21:06:43,036][44959] Updated weights for policy 1, policy_version 27310 (0.0007) [2023-10-12 21:06:43,159][44958] Updated weights for policy 0, policy_version 27160 (0.0009) [2023-10-12 21:06:43,402][44959] Updated weights for policy 1, policy_version 27320 (0.0009) [2023-10-12 21:06:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 55803904. Throughput: 0: 1622.3, 1: 1662.1. Samples: 13964552. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) [2023-10-12 21:06:46,443][43579] Avg episode reward: [(0, '280.210'), (1, '268.210')] [2023-10-12 21:06:47,345][44958] Updated weights for policy 0, policy_version 27170 (0.0008) [2023-10-12 21:06:47,479][44959] Updated weights for policy 1, policy_version 27330 (0.0008) [2023-10-12 21:06:47,709][44958] Updated weights for policy 0, policy_version 27180 (0.0008) [2023-10-12 21:06:47,849][44959] Updated weights for policy 1, policy_version 27340 (0.0008) [2023-10-12 21:06:48,075][44958] Updated weights for policy 0, policy_version 27190 (0.0009) [2023-10-12 21:06:48,209][44959] Updated weights for policy 1, policy_version 27350 (0.0008) [2023-10-12 21:06:48,448][44958] Updated weights for policy 0, policy_version 27200 (0.0010) [2023-10-12 21:06:48,583][44959] Updated weights for policy 1, policy_version 27360 (0.0009) [2023-10-12 21:06:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55869440. Throughput: 0: 1619.8, 1: 1654.2. Samples: 13973458. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) [2023-10-12 21:06:51,443][43579] Avg episode reward: [(0, '278.820'), (1, '272.020')] [2023-10-12 21:06:52,581][44958] Updated weights for policy 0, policy_version 27210 (0.0009) [2023-10-12 21:06:52,895][44959] Updated weights for policy 1, policy_version 27370 (0.0007) [2023-10-12 21:06:52,964][44958] Updated weights for policy 0, policy_version 27220 (0.0007) [2023-10-12 21:06:53,262][44959] Updated weights for policy 1, policy_version 27380 (0.0008) [2023-10-12 21:06:53,332][44958] Updated weights for policy 0, policy_version 27230 (0.0009) [2023-10-12 21:06:53,626][44959] Updated weights for policy 1, policy_version 27390 (0.0009) [2023-10-12 21:06:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 55934976. Throughput: 0: 1629.3, 1: 1656.8. Samples: 13993734. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) [2023-10-12 21:06:56,444][43579] Avg episode reward: [(0, '274.530'), (1, '273.450')] [2023-10-12 21:06:57,518][44958] Updated weights for policy 0, policy_version 27240 (0.0007) [2023-10-12 21:06:57,769][44959] Updated weights for policy 1, policy_version 27400 (0.0009) [2023-10-12 21:06:57,896][44958] Updated weights for policy 0, policy_version 27250 (0.0007) [2023-10-12 21:06:58,137][44959] Updated weights for policy 1, policy_version 27410 (0.0009) [2023-10-12 21:06:58,273][44958] Updated weights for policy 0, policy_version 27260 (0.0007) [2023-10-12 21:06:58,503][44959] Updated weights for policy 1, policy_version 27420 (0.0009) [2023-10-12 21:07:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56000512. Throughput: 0: 1634.9, 1: 1647.3. Samples: 14013990. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:07:01,443][43579] Avg episode reward: [(0, '274.500'), (1, '274.030')] [2023-10-12 21:07:02,434][44958] Updated weights for policy 0, policy_version 27270 (0.0009) [2023-10-12 21:07:02,564][44959] Updated weights for policy 1, policy_version 27430 (0.0009) [2023-10-12 21:07:02,817][44958] Updated weights for policy 0, policy_version 27280 (0.0008) [2023-10-12 21:07:02,938][44959] Updated weights for policy 1, policy_version 27440 (0.0009) [2023-10-12 21:07:03,184][44958] Updated weights for policy 0, policy_version 27290 (0.0008) [2023-10-12 21:07:03,302][44959] Updated weights for policy 1, policy_version 27450 (0.0008) [2023-10-12 21:07:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56066048. Throughput: 0: 1637.4, 1: 1644.8. Samples: 14022860. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:07:06,444][43579] Avg episode reward: [(0, '273.240'), (1, '275.430')] [2023-10-12 21:07:07,546][44959] Updated weights for policy 1, policy_version 27460 (0.0009) [2023-10-12 21:07:07,600][44958] Updated weights for policy 0, policy_version 27300 (0.0009) [2023-10-12 21:07:07,908][44959] Updated weights for policy 1, policy_version 27470 (0.0008) [2023-10-12 21:07:07,966][44958] Updated weights for policy 0, policy_version 27310 (0.0008) [2023-10-12 21:07:08,274][44959] Updated weights for policy 1, policy_version 27480 (0.0009) [2023-10-12 21:07:08,343][44958] Updated weights for policy 0, policy_version 27320 (0.0009) [2023-10-12 21:07:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56131584. Throughput: 0: 1637.3, 1: 1650.0. Samples: 14043256. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:07:11,443][43579] Avg episode reward: [(0, '272.220'), (1, '270.240')] [2023-10-12 21:07:12,440][44959] Updated weights for policy 1, policy_version 27490 (0.0008) [2023-10-12 21:07:12,504][44958] Updated weights for policy 0, policy_version 27330 (0.0009) [2023-10-12 21:07:12,807][44959] Updated weights for policy 1, policy_version 27500 (0.0007) [2023-10-12 21:07:12,882][44958] Updated weights for policy 0, policy_version 27340 (0.0008) [2023-10-12 21:07:13,172][44959] Updated weights for policy 1, policy_version 27510 (0.0008) [2023-10-12 21:07:13,254][44958] Updated weights for policy 0, policy_version 27350 (0.0009) [2023-10-12 21:07:13,535][44959] Updated weights for policy 1, policy_version 27520 (0.0007) [2023-10-12 21:07:13,618][44958] Updated weights for policy 0, policy_version 27360 (0.0009) [2023-10-12 21:07:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 56197120. Throughput: 0: 1635.7, 1: 1647.5. Samples: 14063568. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:07:16,443][43579] Avg episode reward: [(0, '274.330'), (1, '273.900')] [2023-10-12 21:07:17,579][44959] Updated weights for policy 1, policy_version 27530 (0.0007) [2023-10-12 21:07:17,754][44958] Updated weights for policy 0, policy_version 27370 (0.0009) [2023-10-12 21:07:17,950][44959] Updated weights for policy 1, policy_version 27540 (0.0007) [2023-10-12 21:07:18,124][44958] Updated weights for policy 0, policy_version 27380 (0.0008) [2023-10-12 21:07:18,307][44959] Updated weights for policy 1, policy_version 27550 (0.0008) [2023-10-12 21:07:18,484][44958] Updated weights for policy 0, policy_version 27390 (0.0010) [2023-10-12 21:07:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56262656. Throughput: 0: 1638.8, 1: 1647.2. Samples: 14072378. Policy #0 lag: (min: 1.0, avg: 10.6, max: 33.0) [2023-10-12 21:07:21,444][43579] Avg episode reward: [(0, '270.810'), (1, '275.920')] [2023-10-12 21:07:22,451][44959] Updated weights for policy 1, policy_version 27560 (0.0009) [2023-10-12 21:07:22,660][44958] Updated weights for policy 0, policy_version 27400 (0.0009) [2023-10-12 21:07:22,809][44959] Updated weights for policy 1, policy_version 27570 (0.0008) [2023-10-12 21:07:23,033][44958] Updated weights for policy 0, policy_version 27410 (0.0009) [2023-10-12 21:07:23,176][44959] Updated weights for policy 1, policy_version 27580 (0.0009) [2023-10-12 21:07:23,408][44958] Updated weights for policy 0, policy_version 27420 (0.0007) [2023-10-12 21:07:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56328192. Throughput: 0: 1643.4, 1: 1651.1. Samples: 14092778. Policy #0 lag: (min: 1.0, avg: 10.6, max: 33.0) [2023-10-12 21:07:26,444][43579] Avg episode reward: [(0, '267.150'), (1, '271.090')] [2023-10-12 21:07:27,260][44959] Updated weights for policy 1, policy_version 27590 (0.0009) [2023-10-12 21:07:27,408][44958] Updated weights for policy 0, policy_version 27430 (0.0007) [2023-10-12 21:07:27,637][44959] Updated weights for policy 1, policy_version 27600 (0.0007) [2023-10-12 21:07:27,774][44958] Updated weights for policy 0, policy_version 27440 (0.0007) [2023-10-12 21:07:28,002][44959] Updated weights for policy 1, policy_version 27610 (0.0007) [2023-10-12 21:07:28,152][44958] Updated weights for policy 0, policy_version 27450 (0.0007) [2023-10-12 21:07:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56393728. Throughput: 0: 1648.6, 1: 1644.9. Samples: 14112760. Policy #0 lag: (min: 1.0, avg: 10.6, max: 33.0) [2023-10-12 21:07:31,444][43579] Avg episode reward: [(0, '265.600'), (1, '268.200')] [2023-10-12 21:07:32,264][44959] Updated weights for policy 1, policy_version 27620 (0.0007) [2023-10-12 21:07:32,361][44958] Updated weights for policy 0, policy_version 27460 (0.0008) [2023-10-12 21:07:32,634][44959] Updated weights for policy 1, policy_version 27630 (0.0007) [2023-10-12 21:07:32,734][44958] Updated weights for policy 0, policy_version 27470 (0.0007) [2023-10-12 21:07:33,000][44959] Updated weights for policy 1, policy_version 27640 (0.0008) [2023-10-12 21:07:33,104][44958] Updated weights for policy 0, policy_version 27480 (0.0009) [2023-10-12 21:07:36,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56459264. Throughput: 0: 1650.9, 1: 1642.4. Samples: 14121658. Policy #0 lag: (min: 1.0, avg: 10.6, max: 33.0) [2023-10-12 21:07:36,443][43579] Avg episode reward: [(0, '259.090'), (1, '274.100')] [2023-10-12 21:07:37,046][44958] Updated weights for policy 0, policy_version 27490 (0.0010) [2023-10-12 21:07:37,410][44958] Updated weights for policy 0, policy_version 27500 (0.0010) [2023-10-12 21:07:37,479][44959] Updated weights for policy 1, policy_version 27650 (0.0008) [2023-10-12 21:07:37,778][44958] Updated weights for policy 0, policy_version 27510 (0.0009) [2023-10-12 21:07:37,835][44959] Updated weights for policy 1, policy_version 27660 (0.0008) [2023-10-12 21:07:38,156][44958] Updated weights for policy 0, policy_version 27520 (0.0007) [2023-10-12 21:07:38,207][44959] Updated weights for policy 1, policy_version 27670 (0.0010) [2023-10-12 21:07:38,570][44959] Updated weights for policy 1, policy_version 27680 (0.0010) [2023-10-12 21:07:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56524800. Throughput: 0: 1653.2, 1: 1639.9. Samples: 14141920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:07:41,443][43579] Avg episode reward: [(0, '254.160'), (1, '274.940')] [2023-10-12 21:07:42,380][44958] Updated weights for policy 0, policy_version 27530 (0.0008) [2023-10-12 21:07:42,690][44959] Updated weights for policy 1, policy_version 27690 (0.0008) [2023-10-12 21:07:42,752][44958] Updated weights for policy 0, policy_version 27540 (0.0008) [2023-10-12 21:07:43,061][44959] Updated weights for policy 1, policy_version 27700 (0.0010) [2023-10-12 21:07:43,125][44958] Updated weights for policy 0, policy_version 27550 (0.0007) [2023-10-12 21:07:43,433][44959] Updated weights for policy 1, policy_version 27710 (0.0008) [2023-10-12 21:07:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56590336. Throughput: 0: 1653.2, 1: 1646.4. Samples: 14162472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:07:46,443][43579] Avg episode reward: [(0, '254.920'), (1, '278.200')] [2023-10-12 21:07:46,450][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000027552_28213248.pth... [2023-10-12 21:07:46,450][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000027712_28377088.pth... [2023-10-12 21:07:46,487][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000026176_26804224.pth [2023-10-12 21:07:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000026016_26640384.pth [2023-10-12 21:07:47,402][44958] Updated weights for policy 0, policy_version 27560 (0.0008) [2023-10-12 21:07:47,477][44959] Updated weights for policy 1, policy_version 27720 (0.0008) [2023-10-12 21:07:47,773][44958] Updated weights for policy 0, policy_version 27570 (0.0009) [2023-10-12 21:07:47,847][44959] Updated weights for policy 1, policy_version 27730 (0.0008) [2023-10-12 21:07:48,143][44958] Updated weights for policy 0, policy_version 27580 (0.0008) [2023-10-12 21:07:48,218][44959] Updated weights for policy 1, policy_version 27740 (0.0009) [2023-10-12 21:07:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56655872. Throughput: 0: 1649.6, 1: 1649.0. Samples: 14171296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:07:51,443][43579] Avg episode reward: [(0, '252.480'), (1, '276.080')] [2023-10-12 21:07:52,161][44958] Updated weights for policy 0, policy_version 27590 (0.0008) [2023-10-12 21:07:52,362][44959] Updated weights for policy 1, policy_version 27750 (0.0009) [2023-10-12 21:07:52,534][44958] Updated weights for policy 0, policy_version 27600 (0.0007) [2023-10-12 21:07:52,735][44959] Updated weights for policy 1, policy_version 27760 (0.0008) [2023-10-12 21:07:52,917][44958] Updated weights for policy 0, policy_version 27610 (0.0008) [2023-10-12 21:07:53,106][44959] Updated weights for policy 1, policy_version 27770 (0.0007) [2023-10-12 21:07:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 56721408. Throughput: 0: 1653.8, 1: 1640.0. Samples: 14191478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:07:56,443][43579] Avg episode reward: [(0, '253.590'), (1, '274.530')] [2023-10-12 21:07:56,968][44958] Updated weights for policy 0, policy_version 27620 (0.0008) [2023-10-12 21:07:57,319][44959] Updated weights for policy 1, policy_version 27780 (0.0010) [2023-10-12 21:07:57,346][44958] Updated weights for policy 0, policy_version 27630 (0.0007) [2023-10-12 21:07:57,678][44959] Updated weights for policy 1, policy_version 27790 (0.0009) [2023-10-12 21:07:57,713][44958] Updated weights for policy 0, policy_version 27640 (0.0009) [2023-10-12 21:07:58,048][44959] Updated weights for policy 1, policy_version 27800 (0.0010) [2023-10-12 21:08:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56786944. Throughput: 0: 1657.2, 1: 1640.4. Samples: 14211960. Policy #0 lag: (min: 18.0, avg: 18.9, max: 39.0) [2023-10-12 21:08:01,443][43579] Avg episode reward: [(0, '252.320'), (1, '273.570')] [2023-10-12 21:08:01,900][44958] Updated weights for policy 0, policy_version 27650 (0.0010) [2023-10-12 21:08:02,077][44959] Updated weights for policy 1, policy_version 27810 (0.0009) [2023-10-12 21:08:02,264][44958] Updated weights for policy 0, policy_version 27660 (0.0009) [2023-10-12 21:08:02,447][44959] Updated weights for policy 1, policy_version 27820 (0.0010) [2023-10-12 21:08:02,646][44958] Updated weights for policy 0, policy_version 27670 (0.0009) [2023-10-12 21:08:02,817][44959] Updated weights for policy 1, policy_version 27830 (0.0009) [2023-10-12 21:08:03,018][44958] Updated weights for policy 0, policy_version 27680 (0.0008) [2023-10-12 21:08:03,187][44959] Updated weights for policy 1, policy_version 27840 (0.0009) [2023-10-12 21:08:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 56852480. Throughput: 0: 1651.6, 1: 1646.1. Samples: 14220776. Policy #0 lag: (min: 18.0, avg: 18.9, max: 39.0) [2023-10-12 21:08:06,444][43579] Avg episode reward: [(0, '258.880'), (1, '273.950')] [2023-10-12 21:08:07,193][44958] Updated weights for policy 0, policy_version 27690 (0.0008) [2023-10-12 21:08:07,324][44959] Updated weights for policy 1, policy_version 27850 (0.0007) [2023-10-12 21:08:07,553][44958] Updated weights for policy 0, policy_version 27700 (0.0008) [2023-10-12 21:08:07,693][44959] Updated weights for policy 1, policy_version 27860 (0.0008) [2023-10-12 21:08:07,930][44958] Updated weights for policy 0, policy_version 27710 (0.0008) [2023-10-12 21:08:08,063][44959] Updated weights for policy 1, policy_version 27870 (0.0007) [2023-10-12 21:08:11,444][43579] Fps is (10 sec: 13104.7, 60 sec: 13106.8, 300 sec: 13107.1). Total num frames: 56918016. Throughput: 0: 1646.2, 1: 1646.5. Samples: 14240954. Policy #0 lag: (min: 18.0, avg: 18.9, max: 39.0) [2023-10-12 21:08:11,445][43579] Avg episode reward: [(0, '259.460'), (1, '271.620')] [2023-10-12 21:08:12,183][44958] Updated weights for policy 0, policy_version 27720 (0.0008) [2023-10-12 21:08:12,328][44959] Updated weights for policy 1, policy_version 27880 (0.0008) [2023-10-12 21:08:12,556][44958] Updated weights for policy 0, policy_version 27730 (0.0008) [2023-10-12 21:08:12,711][44959] Updated weights for policy 1, policy_version 27890 (0.0008) [2023-10-12 21:08:12,925][44958] Updated weights for policy 0, policy_version 27740 (0.0007) [2023-10-12 21:08:13,081][44959] Updated weights for policy 1, policy_version 27900 (0.0008) [2023-10-12 21:08:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 56983552. Throughput: 0: 1645.2, 1: 1655.5. Samples: 14261290. Policy #0 lag: (min: 18.0, avg: 18.9, max: 39.0) [2023-10-12 21:08:16,444][43579] Avg episode reward: [(0, '258.180'), (1, '262.550')] [2023-10-12 21:08:17,093][44959] Updated weights for policy 1, policy_version 27910 (0.0009) [2023-10-12 21:08:17,143][44958] Updated weights for policy 0, policy_version 27750 (0.0008) [2023-10-12 21:08:17,458][44959] Updated weights for policy 1, policy_version 27920 (0.0007) [2023-10-12 21:08:17,509][44958] Updated weights for policy 0, policy_version 27760 (0.0008) [2023-10-12 21:08:17,833][44959] Updated weights for policy 1, policy_version 27930 (0.0007) [2023-10-12 21:08:17,884][44958] Updated weights for policy 0, policy_version 27770 (0.0008) [2023-10-12 21:08:21,443][43579] Fps is (10 sec: 13109.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57049088. Throughput: 0: 1640.1, 1: 1656.7. Samples: 14270014. Policy #0 lag: (min: 15.0, avg: 16.9, max: 45.0) [2023-10-12 21:08:21,444][43579] Avg episode reward: [(0, '262.470'), (1, '266.140')] [2023-10-12 21:08:21,967][44958] Updated weights for policy 0, policy_version 27780 (0.0008) [2023-10-12 21:08:22,050][44959] Updated weights for policy 1, policy_version 27940 (0.0007) [2023-10-12 21:08:22,342][44958] Updated weights for policy 0, policy_version 27790 (0.0007) [2023-10-12 21:08:22,413][44959] Updated weights for policy 1, policy_version 27950 (0.0007) [2023-10-12 21:08:22,709][44958] Updated weights for policy 0, policy_version 27800 (0.0007) [2023-10-12 21:08:22,780][44959] Updated weights for policy 1, policy_version 27960 (0.0009) [2023-10-12 21:08:26,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57114624. Throughput: 0: 1641.3, 1: 1661.0. Samples: 14290522. Policy #0 lag: (min: 15.0, avg: 16.9, max: 45.0) [2023-10-12 21:08:26,444][43579] Avg episode reward: [(0, '264.380'), (1, '270.270')] [2023-10-12 21:08:26,782][44959] Updated weights for policy 1, policy_version 27970 (0.0009) [2023-10-12 21:08:27,087][44958] Updated weights for policy 0, policy_version 27810 (0.0008) [2023-10-12 21:08:27,155][44959] Updated weights for policy 1, policy_version 27980 (0.0008) [2023-10-12 21:08:27,458][44958] Updated weights for policy 0, policy_version 27820 (0.0009) [2023-10-12 21:08:27,514][44959] Updated weights for policy 1, policy_version 27990 (0.0008) [2023-10-12 21:08:27,824][44958] Updated weights for policy 0, policy_version 27830 (0.0008) [2023-10-12 21:08:27,889][44959] Updated weights for policy 1, policy_version 28000 (0.0008) [2023-10-12 21:08:28,200][44958] Updated weights for policy 0, policy_version 27840 (0.0009) [2023-10-12 21:08:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 57180160. Throughput: 0: 1633.4, 1: 1657.0. Samples: 14310542. Policy #0 lag: (min: 15.0, avg: 16.9, max: 45.0) [2023-10-12 21:08:31,443][43579] Avg episode reward: [(0, '264.540'), (1, '271.050')] [2023-10-12 21:08:32,110][44959] Updated weights for policy 1, policy_version 28010 (0.0008) [2023-10-12 21:08:32,481][44959] Updated weights for policy 1, policy_version 28020 (0.0008) [2023-10-12 21:08:32,623][44958] Updated weights for policy 0, policy_version 27850 (0.0007) [2023-10-12 21:08:32,850][44959] Updated weights for policy 1, policy_version 28030 (0.0007) [2023-10-12 21:08:32,997][44958] Updated weights for policy 0, policy_version 27860 (0.0007) [2023-10-12 21:08:33,374][44958] Updated weights for policy 0, policy_version 27870 (0.0008) [2023-10-12 21:08:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57245696. Throughput: 0: 1629.6, 1: 1657.6. Samples: 14319222. Policy #0 lag: (min: 15.0, avg: 16.9, max: 45.0) [2023-10-12 21:08:36,444][43579] Avg episode reward: [(0, '260.670'), (1, '267.490')] [2023-10-12 21:08:37,040][44959] Updated weights for policy 1, policy_version 28040 (0.0008) [2023-10-12 21:08:37,306][44958] Updated weights for policy 0, policy_version 27880 (0.0007) [2023-10-12 21:08:37,401][44959] Updated weights for policy 1, policy_version 28050 (0.0007) [2023-10-12 21:08:37,679][44958] Updated weights for policy 0, policy_version 27890 (0.0007) [2023-10-12 21:08:37,769][44959] Updated weights for policy 1, policy_version 28060 (0.0010) [2023-10-12 21:08:38,037][44958] Updated weights for policy 0, policy_version 27900 (0.0009) [2023-10-12 21:08:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57311232. Throughput: 0: 1634.2, 1: 1658.5. Samples: 14339652. Policy #0 lag: (min: 25.0, avg: 29.2, max: 57.0) [2023-10-12 21:08:41,444][43579] Avg episode reward: [(0, '259.730'), (1, '269.650')] [2023-10-12 21:08:41,970][44959] Updated weights for policy 1, policy_version 28070 (0.0009) [2023-10-12 21:08:42,219][44958] Updated weights for policy 0, policy_version 27910 (0.0008) [2023-10-12 21:08:42,328][44959] Updated weights for policy 1, policy_version 28080 (0.0010) [2023-10-12 21:08:42,591][44958] Updated weights for policy 0, policy_version 27920 (0.0007) [2023-10-12 21:08:42,692][44959] Updated weights for policy 1, policy_version 28090 (0.0009) [2023-10-12 21:08:42,960][44958] Updated weights for policy 0, policy_version 27930 (0.0009) [2023-10-12 21:08:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57376768. Throughput: 0: 1629.8, 1: 1653.7. Samples: 14359718. Policy #0 lag: (min: 25.0, avg: 29.2, max: 57.0) [2023-10-12 21:08:46,443][43579] Avg episode reward: [(0, '259.430'), (1, '275.590')] [2023-10-12 21:08:46,914][44959] Updated weights for policy 1, policy_version 28100 (0.0008) [2023-10-12 21:08:47,283][44959] Updated weights for policy 1, policy_version 28110 (0.0007) [2023-10-12 21:08:47,310][44958] Updated weights for policy 0, policy_version 27940 (0.0009) [2023-10-12 21:08:47,647][44959] Updated weights for policy 1, policy_version 28120 (0.0008) [2023-10-12 21:08:47,686][44958] Updated weights for policy 0, policy_version 27950 (0.0007) [2023-10-12 21:08:48,056][44958] Updated weights for policy 0, policy_version 27960 (0.0007) [2023-10-12 21:08:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57442304. Throughput: 0: 1633.6, 1: 1649.0. Samples: 14368492. Policy #0 lag: (min: 25.0, avg: 29.2, max: 57.0) [2023-10-12 21:08:51,444][43579] Avg episode reward: [(0, '256.820'), (1, '274.730')] [2023-10-12 21:08:51,959][44958] Updated weights for policy 0, policy_version 27970 (0.0009) [2023-10-12 21:08:52,105][44959] Updated weights for policy 1, policy_version 28130 (0.0008) [2023-10-12 21:08:52,335][44958] Updated weights for policy 0, policy_version 27980 (0.0009) [2023-10-12 21:08:52,483][44959] Updated weights for policy 1, policy_version 28140 (0.0008) [2023-10-12 21:08:52,703][44958] Updated weights for policy 0, policy_version 27990 (0.0007) [2023-10-12 21:08:52,851][44959] Updated weights for policy 1, policy_version 28150 (0.0007) [2023-10-12 21:08:53,074][44958] Updated weights for policy 0, policy_version 28000 (0.0007) [2023-10-12 21:08:53,220][44959] Updated weights for policy 1, policy_version 28160 (0.0007) [2023-10-12 21:08:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57507840. Throughput: 0: 1639.5, 1: 1645.9. Samples: 14388792. Policy #0 lag: (min: 25.0, avg: 29.2, max: 57.0) [2023-10-12 21:08:56,443][43579] Avg episode reward: [(0, '256.020'), (1, '273.840')] [2023-10-12 21:08:57,287][44959] Updated weights for policy 1, policy_version 28170 (0.0007) [2023-10-12 21:08:57,384][44958] Updated weights for policy 0, policy_version 28010 (0.0007) [2023-10-12 21:08:57,648][44959] Updated weights for policy 1, policy_version 28180 (0.0009) [2023-10-12 21:08:57,750][44958] Updated weights for policy 0, policy_version 28020 (0.0008) [2023-10-12 21:08:58,013][44959] Updated weights for policy 1, policy_version 28190 (0.0008) [2023-10-12 21:08:58,132][44958] Updated weights for policy 0, policy_version 28030 (0.0009) [2023-10-12 21:09:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57573376. Throughput: 0: 1633.5, 1: 1641.8. Samples: 14408678. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-12 21:09:01,444][43579] Avg episode reward: [(0, '258.190'), (1, '273.690')] [2023-10-12 21:09:02,146][44959] Updated weights for policy 1, policy_version 28200 (0.0007) [2023-10-12 21:09:02,383][44958] Updated weights for policy 0, policy_version 28040 (0.0010) [2023-10-12 21:09:02,522][44959] Updated weights for policy 1, policy_version 28210 (0.0008) [2023-10-12 21:09:02,744][44958] Updated weights for policy 0, policy_version 28050 (0.0008) [2023-10-12 21:09:02,895][44959] Updated weights for policy 1, policy_version 28220 (0.0008) [2023-10-12 21:09:03,125][44958] Updated weights for policy 0, policy_version 28060 (0.0009) [2023-10-12 21:09:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57638912. Throughput: 0: 1636.8, 1: 1638.4. Samples: 14417396. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-12 21:09:06,444][43579] Avg episode reward: [(0, '257.470'), (1, '266.270')] [2023-10-12 21:09:07,062][44959] Updated weights for policy 1, policy_version 28230 (0.0008) [2023-10-12 21:09:07,115][44958] Updated weights for policy 0, policy_version 28070 (0.0008) [2023-10-12 21:09:07,430][44959] Updated weights for policy 1, policy_version 28240 (0.0008) [2023-10-12 21:09:07,477][44958] Updated weights for policy 0, policy_version 28080 (0.0009) [2023-10-12 21:09:07,793][44959] Updated weights for policy 1, policy_version 28250 (0.0009) [2023-10-12 21:09:07,860][44958] Updated weights for policy 0, policy_version 28090 (0.0007) [2023-10-12 21:09:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.6, 300 sec: 13107.2). Total num frames: 57704448. Throughput: 0: 1635.0, 1: 1641.7. Samples: 14437974. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-12 21:09:11,443][43579] Avg episode reward: [(0, '260.010'), (1, '269.460')] [2023-10-12 21:09:11,849][44959] Updated weights for policy 1, policy_version 28260 (0.0008) [2023-10-12 21:09:12,026][44958] Updated weights for policy 0, policy_version 28100 (0.0009) [2023-10-12 21:09:12,224][44959] Updated weights for policy 1, policy_version 28270 (0.0008) [2023-10-12 21:09:12,403][44958] Updated weights for policy 0, policy_version 28110 (0.0007) [2023-10-12 21:09:12,590][44959] Updated weights for policy 1, policy_version 28280 (0.0007) [2023-10-12 21:09:12,770][44958] Updated weights for policy 0, policy_version 28120 (0.0009) [2023-10-12 21:09:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57769984. Throughput: 0: 1641.5, 1: 1635.6. Samples: 14458014. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-12 21:09:16,444][43579] Avg episode reward: [(0, '262.400'), (1, '273.020')] [2023-10-12 21:09:16,869][44959] Updated weights for policy 1, policy_version 28290 (0.0008) [2023-10-12 21:09:16,973][44958] Updated weights for policy 0, policy_version 28130 (0.0008) [2023-10-12 21:09:17,235][44959] Updated weights for policy 1, policy_version 28300 (0.0009) [2023-10-12 21:09:17,396][44958] Updated weights for policy 0, policy_version 28140 (0.0009) [2023-10-12 21:09:17,609][44959] Updated weights for policy 1, policy_version 28310 (0.0009) [2023-10-12 21:09:17,766][44958] Updated weights for policy 0, policy_version 28150 (0.0010) [2023-10-12 21:09:17,972][44959] Updated weights for policy 1, policy_version 28320 (0.0009) [2023-10-12 21:09:18,131][44958] Updated weights for policy 0, policy_version 28160 (0.0008) [2023-10-12 21:09:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57835520. Throughput: 0: 1644.0, 1: 1631.4. Samples: 14466616. Policy #0 lag: (min: 1.0, avg: 8.7, max: 33.0) [2023-10-12 21:09:21,443][43579] Avg episode reward: [(0, '266.000'), (1, '273.740')] [2023-10-12 21:09:22,076][44959] Updated weights for policy 1, policy_version 28330 (0.0007) [2023-10-12 21:09:22,427][44958] Updated weights for policy 0, policy_version 28170 (0.0007) [2023-10-12 21:09:22,444][44959] Updated weights for policy 1, policy_version 28340 (0.0007) [2023-10-12 21:09:22,803][44958] Updated weights for policy 0, policy_version 28180 (0.0010) [2023-10-12 21:09:22,806][44959] Updated weights for policy 1, policy_version 28350 (0.0010) [2023-10-12 21:09:23,177][44958] Updated weights for policy 0, policy_version 28190 (0.0009) [2023-10-12 21:09:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 57901056. Throughput: 0: 1634.4, 1: 1637.6. Samples: 14486892. Policy #0 lag: (min: 1.0, avg: 8.7, max: 33.0) [2023-10-12 21:09:26,444][43579] Avg episode reward: [(0, '262.490'), (1, '280.550')] [2023-10-12 21:09:26,973][44959] Updated weights for policy 1, policy_version 28360 (0.0010) [2023-10-12 21:09:27,345][44959] Updated weights for policy 1, policy_version 28370 (0.0009) [2023-10-12 21:09:27,348][44958] Updated weights for policy 0, policy_version 28200 (0.0007) [2023-10-12 21:09:27,716][44959] Updated weights for policy 1, policy_version 28380 (0.0008) [2023-10-12 21:09:27,726][44958] Updated weights for policy 0, policy_version 28210 (0.0009) [2023-10-12 21:09:28,091][44958] Updated weights for policy 0, policy_version 28220 (0.0008) [2023-10-12 21:09:31,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 57966592. Throughput: 0: 1635.4, 1: 1637.2. Samples: 14506988. Policy #0 lag: (min: 1.0, avg: 8.7, max: 33.0) [2023-10-12 21:09:31,444][43579] Avg episode reward: [(0, '265.530'), (1, '277.670')] [2023-10-12 21:09:31,879][44959] Updated weights for policy 1, policy_version 28390 (0.0008) [2023-10-12 21:09:32,188][44958] Updated weights for policy 0, policy_version 28230 (0.0008) [2023-10-12 21:09:32,241][44959] Updated weights for policy 1, policy_version 28400 (0.0010) [2023-10-12 21:09:32,552][44958] Updated weights for policy 0, policy_version 28240 (0.0007) [2023-10-12 21:09:32,607][44959] Updated weights for policy 1, policy_version 28410 (0.0007) [2023-10-12 21:09:32,928][44958] Updated weights for policy 0, policy_version 28250 (0.0008) [2023-10-12 21:09:36,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58032128. Throughput: 0: 1635.8, 1: 1639.6. Samples: 14515884. Policy #0 lag: (min: 1.0, avg: 8.7, max: 33.0) [2023-10-12 21:09:36,443][43579] Avg episode reward: [(0, '266.270'), (1, '281.340')] [2023-10-12 21:09:36,855][44959] Updated weights for policy 1, policy_version 28420 (0.0007) [2023-10-12 21:09:37,062][44958] Updated weights for policy 0, policy_version 28260 (0.0008) [2023-10-12 21:09:37,229][44959] Updated weights for policy 1, policy_version 28430 (0.0008) [2023-10-12 21:09:37,440][44958] Updated weights for policy 0, policy_version 28270 (0.0008) [2023-10-12 21:09:37,598][44959] Updated weights for policy 1, policy_version 28440 (0.0010) [2023-10-12 21:09:37,818][44958] Updated weights for policy 0, policy_version 28280 (0.0007) [2023-10-12 21:09:41,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58097664. Throughput: 0: 1635.8, 1: 1642.7. Samples: 14536322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:09:41,443][43579] Avg episode reward: [(0, '264.740'), (1, '275.400')] [2023-10-12 21:09:41,768][44959] Updated weights for policy 1, policy_version 28450 (0.0008) [2023-10-12 21:09:42,143][44959] Updated weights for policy 1, policy_version 28460 (0.0009) [2023-10-12 21:09:42,270][44958] Updated weights for policy 0, policy_version 28290 (0.0008) [2023-10-12 21:09:42,517][44959] Updated weights for policy 1, policy_version 28470 (0.0009) [2023-10-12 21:09:42,635][44958] Updated weights for policy 0, policy_version 28300 (0.0008) [2023-10-12 21:09:42,887][44959] Updated weights for policy 1, policy_version 28480 (0.0009) [2023-10-12 21:09:43,010][44958] Updated weights for policy 0, policy_version 28310 (0.0008) [2023-10-12 21:09:43,374][44958] Updated weights for policy 0, policy_version 28320 (0.0008) [2023-10-12 21:09:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58163200. Throughput: 0: 1638.1, 1: 1644.4. Samples: 14556390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:09:46,443][43579] Avg episode reward: [(0, '258.130'), (1, '275.490')] [2023-10-12 21:09:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000028480_29163520.pth... [2023-10-12 21:09:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000028320_28999680.pth... [2023-10-12 21:09:46,491][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000026784_27426816.pth [2023-10-12 21:09:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000026944_27590656.pth [2023-10-12 21:09:47,054][44959] Updated weights for policy 1, policy_version 28490 (0.0008) [2023-10-12 21:09:47,401][44958] Updated weights for policy 0, policy_version 28330 (0.0008) [2023-10-12 21:09:47,435][44959] Updated weights for policy 1, policy_version 28500 (0.0008) [2023-10-12 21:09:47,774][44958] Updated weights for policy 0, policy_version 28340 (0.0008) [2023-10-12 21:09:47,802][44959] Updated weights for policy 1, policy_version 28510 (0.0008) [2023-10-12 21:09:48,143][44958] Updated weights for policy 0, policy_version 28350 (0.0008) [2023-10-12 21:09:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58228736. Throughput: 0: 1640.8, 1: 1646.4. Samples: 14565318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:09:51,443][43579] Avg episode reward: [(0, '258.860'), (1, '266.810')] [2023-10-12 21:09:51,892][44959] Updated weights for policy 1, policy_version 28520 (0.0007) [2023-10-12 21:09:52,255][44959] Updated weights for policy 1, policy_version 28530 (0.0008) [2023-10-12 21:09:52,348][44958] Updated weights for policy 0, policy_version 28360 (0.0009) [2023-10-12 21:09:52,624][44959] Updated weights for policy 1, policy_version 28540 (0.0009) [2023-10-12 21:09:52,713][44958] Updated weights for policy 0, policy_version 28370 (0.0009) [2023-10-12 21:09:53,090][44958] Updated weights for policy 0, policy_version 28380 (0.0009) [2023-10-12 21:09:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58294272. Throughput: 0: 1639.4, 1: 1640.4. Samples: 14585566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:09:56,443][43579] Avg episode reward: [(0, '262.810'), (1, '267.980')] [2023-10-12 21:09:56,814][44959] Updated weights for policy 1, policy_version 28550 (0.0008) [2023-10-12 21:09:57,178][44959] Updated weights for policy 1, policy_version 28560 (0.0009) [2023-10-12 21:09:57,299][44958] Updated weights for policy 0, policy_version 28390 (0.0007) [2023-10-12 21:09:57,544][44959] Updated weights for policy 1, policy_version 28570 (0.0010) [2023-10-12 21:09:57,662][44958] Updated weights for policy 0, policy_version 28400 (0.0008) [2023-10-12 21:09:58,040][44958] Updated weights for policy 0, policy_version 28410 (0.0009) [2023-10-12 21:10:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 58359808. Throughput: 0: 1638.1, 1: 1643.8. Samples: 14605702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:10:01,443][43579] Avg episode reward: [(0, '267.610'), (1, '270.390')] [2023-10-12 21:10:01,722][44959] Updated weights for policy 1, policy_version 28580 (0.0009) [2023-10-12 21:10:02,097][44959] Updated weights for policy 1, policy_version 28590 (0.0008) [2023-10-12 21:10:02,291][44958] Updated weights for policy 0, policy_version 28420 (0.0008) [2023-10-12 21:10:02,462][44959] Updated weights for policy 1, policy_version 28600 (0.0007) [2023-10-12 21:10:02,681][44958] Updated weights for policy 0, policy_version 28430 (0.0009) [2023-10-12 21:10:03,053][44958] Updated weights for policy 0, policy_version 28440 (0.0009) [2023-10-12 21:10:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58425344. Throughput: 0: 1637.6, 1: 1649.5. Samples: 14614538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:10:06,444][43579] Avg episode reward: [(0, '268.950'), (1, '269.420')] [2023-10-12 21:10:06,543][44959] Updated weights for policy 1, policy_version 28610 (0.0007) [2023-10-12 21:10:06,914][44959] Updated weights for policy 1, policy_version 28620 (0.0009) [2023-10-12 21:10:07,279][44959] Updated weights for policy 1, policy_version 28630 (0.0009) [2023-10-12 21:10:07,358][44958] Updated weights for policy 0, policy_version 28450 (0.0007) [2023-10-12 21:10:07,651][44959] Updated weights for policy 1, policy_version 28640 (0.0007) [2023-10-12 21:10:07,729][44958] Updated weights for policy 0, policy_version 28460 (0.0007) [2023-10-12 21:10:08,091][44958] Updated weights for policy 0, policy_version 28470 (0.0008) [2023-10-12 21:10:08,464][44958] Updated weights for policy 0, policy_version 28480 (0.0007) [2023-10-12 21:10:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58490880. Throughput: 0: 1634.8, 1: 1643.7. Samples: 14634426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:10:11,443][43579] Avg episode reward: [(0, '270.680'), (1, '275.070')] [2023-10-12 21:10:11,902][44959] Updated weights for policy 1, policy_version 28650 (0.0011) [2023-10-12 21:10:12,277][44959] Updated weights for policy 1, policy_version 28660 (0.0011) [2023-10-12 21:10:12,655][44959] Updated weights for policy 1, policy_version 28670 (0.0008) [2023-10-12 21:10:12,728][44958] Updated weights for policy 0, policy_version 28490 (0.0010) [2023-10-12 21:10:13,095][44958] Updated weights for policy 0, policy_version 28500 (0.0009) [2023-10-12 21:10:13,474][44958] Updated weights for policy 0, policy_version 28510 (0.0009) [2023-10-12 21:10:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 58556416. Throughput: 0: 1633.6, 1: 1644.2. Samples: 14654490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:10:16,443][43579] Avg episode reward: [(0, '276.070'), (1, '273.650')] [2023-10-12 21:10:16,683][44959] Updated weights for policy 1, policy_version 28680 (0.0008) [2023-10-12 21:10:17,042][44959] Updated weights for policy 1, policy_version 28690 (0.0008) [2023-10-12 21:10:17,407][44959] Updated weights for policy 1, policy_version 28700 (0.0007) [2023-10-12 21:10:17,669][44958] Updated weights for policy 0, policy_version 28520 (0.0008) [2023-10-12 21:10:18,042][44958] Updated weights for policy 0, policy_version 28530 (0.0010) [2023-10-12 21:10:18,422][44958] Updated weights for policy 0, policy_version 28540 (0.0009) [2023-10-12 21:10:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58621952. Throughput: 0: 1631.8, 1: 1645.1. Samples: 14663344. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:10:21,443][43579] Avg episode reward: [(0, '274.070'), (1, '278.670')] [2023-10-12 21:10:21,663][44959] Updated weights for policy 1, policy_version 28710 (0.0009) [2023-10-12 21:10:22,029][44959] Updated weights for policy 1, policy_version 28720 (0.0011) [2023-10-12 21:10:22,398][44959] Updated weights for policy 1, policy_version 28730 (0.0010) [2023-10-12 21:10:22,579][44958] Updated weights for policy 0, policy_version 28550 (0.0010) [2023-10-12 21:10:22,950][44958] Updated weights for policy 0, policy_version 28560 (0.0009) [2023-10-12 21:10:23,326][44958] Updated weights for policy 0, policy_version 28570 (0.0010) [2023-10-12 21:10:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58687488. Throughput: 0: 1635.3, 1: 1644.2. Samples: 14683900. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:10:26,444][43579] Avg episode reward: [(0, '271.370'), (1, '281.230')] [2023-10-12 21:10:26,576][44959] Updated weights for policy 1, policy_version 28740 (0.0009) [2023-10-12 21:10:26,981][44959] Updated weights for policy 1, policy_version 28750 (0.0008) [2023-10-12 21:10:27,280][44958] Updated weights for policy 0, policy_version 28580 (0.0008) [2023-10-12 21:10:27,342][44959] Updated weights for policy 1, policy_version 28760 (0.0008) [2023-10-12 21:10:27,653][44958] Updated weights for policy 0, policy_version 28590 (0.0009) [2023-10-12 21:10:28,035][44958] Updated weights for policy 0, policy_version 28600 (0.0011) [2023-10-12 21:10:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 58753024. Throughput: 0: 1639.1, 1: 1636.8. Samples: 14703808. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:10:31,443][43579] Avg episode reward: [(0, '266.700'), (1, '277.750')] [2023-10-12 21:10:31,546][44959] Updated weights for policy 1, policy_version 28770 (0.0008) [2023-10-12 21:10:31,923][44959] Updated weights for policy 1, policy_version 28780 (0.0009) [2023-10-12 21:10:32,195][44958] Updated weights for policy 0, policy_version 28610 (0.0010) [2023-10-12 21:10:32,296][44959] Updated weights for policy 1, policy_version 28790 (0.0008) [2023-10-12 21:10:32,562][44958] Updated weights for policy 0, policy_version 28620 (0.0007) [2023-10-12 21:10:32,667][44959] Updated weights for policy 1, policy_version 28800 (0.0007) [2023-10-12 21:10:32,938][44958] Updated weights for policy 0, policy_version 28630 (0.0008) [2023-10-12 21:10:33,308][44958] Updated weights for policy 0, policy_version 28640 (0.0007) [2023-10-12 21:10:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 58818560. Throughput: 0: 1636.8, 1: 1638.1. Samples: 14712692. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:10:36,444][43579] Avg episode reward: [(0, '261.340'), (1, '278.320')] [2023-10-12 21:10:36,904][44959] Updated weights for policy 1, policy_version 28810 (0.0009) [2023-10-12 21:10:37,275][44959] Updated weights for policy 1, policy_version 28820 (0.0008) [2023-10-12 21:10:37,620][44958] Updated weights for policy 0, policy_version 28650 (0.0008) [2023-10-12 21:10:37,640][44959] Updated weights for policy 1, policy_version 28830 (0.0007) [2023-10-12 21:10:37,989][44958] Updated weights for policy 0, policy_version 28660 (0.0009) [2023-10-12 21:10:38,368][44958] Updated weights for policy 0, policy_version 28670 (0.0010) [2023-10-12 21:10:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58884096. Throughput: 0: 1636.2, 1: 1636.8. Samples: 14732850. Policy #0 lag: (min: 3.0, avg: 5.8, max: 32.0) [2023-10-12 21:10:41,443][43579] Avg episode reward: [(0, '262.550'), (1, '274.080')] [2023-10-12 21:10:41,896][44959] Updated weights for policy 1, policy_version 28840 (0.0009) [2023-10-12 21:10:42,262][44959] Updated weights for policy 1, policy_version 28850 (0.0007) [2023-10-12 21:10:42,474][44958] Updated weights for policy 0, policy_version 28680 (0.0007) [2023-10-12 21:10:42,626][44959] Updated weights for policy 1, policy_version 28860 (0.0008) [2023-10-12 21:10:42,842][44958] Updated weights for policy 0, policy_version 28690 (0.0009) [2023-10-12 21:10:43,214][44958] Updated weights for policy 0, policy_version 28700 (0.0009) [2023-10-12 21:10:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 58949632. Throughput: 0: 1633.9, 1: 1632.1. Samples: 14752674. Policy #0 lag: (min: 3.0, avg: 5.8, max: 32.0) [2023-10-12 21:10:46,444][43579] Avg episode reward: [(0, '260.780'), (1, '276.390')] [2023-10-12 21:10:46,834][44959] Updated weights for policy 1, policy_version 28870 (0.0008) [2023-10-12 21:10:47,209][44959] Updated weights for policy 1, policy_version 28880 (0.0009) [2023-10-12 21:10:47,419][44958] Updated weights for policy 0, policy_version 28710 (0.0008) [2023-10-12 21:10:47,582][44959] Updated weights for policy 1, policy_version 28890 (0.0008) [2023-10-12 21:10:47,795][44958] Updated weights for policy 0, policy_version 28720 (0.0008) [2023-10-12 21:10:48,160][44958] Updated weights for policy 0, policy_version 28730 (0.0008) [2023-10-12 21:10:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59015168. Throughput: 0: 1635.6, 1: 1629.4. Samples: 14761466. Policy #0 lag: (min: 3.0, avg: 5.8, max: 32.0) [2023-10-12 21:10:51,443][43579] Avg episode reward: [(0, '261.380'), (1, '267.750')] [2023-10-12 21:10:51,821][44959] Updated weights for policy 1, policy_version 28900 (0.0009) [2023-10-12 21:10:52,193][44959] Updated weights for policy 1, policy_version 28910 (0.0008) [2023-10-12 21:10:52,221][44958] Updated weights for policy 0, policy_version 28740 (0.0008) [2023-10-12 21:10:52,563][44959] Updated weights for policy 1, policy_version 28920 (0.0008) [2023-10-12 21:10:52,594][44958] Updated weights for policy 0, policy_version 28750 (0.0009) [2023-10-12 21:10:52,976][44958] Updated weights for policy 0, policy_version 28760 (0.0008) [2023-10-12 21:10:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59080704. Throughput: 0: 1644.4, 1: 1629.5. Samples: 14781752. Policy #0 lag: (min: 3.0, avg: 5.8, max: 32.0) [2023-10-12 21:10:56,444][43579] Avg episode reward: [(0, '260.580'), (1, '263.880')] [2023-10-12 21:10:56,837][44959] Updated weights for policy 1, policy_version 28930 (0.0008) [2023-10-12 21:10:56,984][44958] Updated weights for policy 0, policy_version 28770 (0.0011) [2023-10-12 21:10:57,200][44959] Updated weights for policy 1, policy_version 28940 (0.0009) [2023-10-12 21:10:57,354][44958] Updated weights for policy 0, policy_version 28780 (0.0007) [2023-10-12 21:10:57,566][44959] Updated weights for policy 1, policy_version 28950 (0.0009) [2023-10-12 21:10:57,731][44958] Updated weights for policy 0, policy_version 28790 (0.0010) [2023-10-12 21:10:57,935][44959] Updated weights for policy 1, policy_version 28960 (0.0009) [2023-10-12 21:10:58,094][44958] Updated weights for policy 0, policy_version 28800 (0.0010) [2023-10-12 21:11:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59146240. Throughput: 0: 1647.3, 1: 1631.0. Samples: 14802014. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-12 21:11:01,443][43579] Avg episode reward: [(0, '263.490'), (1, '262.790')] [2023-10-12 21:11:02,086][44958] Updated weights for policy 0, policy_version 28810 (0.0009) [2023-10-12 21:11:02,124][44959] Updated weights for policy 1, policy_version 28970 (0.0010) [2023-10-12 21:11:02,460][44958] Updated weights for policy 0, policy_version 28820 (0.0009) [2023-10-12 21:11:02,501][44959] Updated weights for policy 1, policy_version 28980 (0.0008) [2023-10-12 21:11:02,828][44958] Updated weights for policy 0, policy_version 28830 (0.0009) [2023-10-12 21:11:02,868][44959] Updated weights for policy 1, policy_version 28990 (0.0009) [2023-10-12 21:11:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59211776. Throughput: 0: 1651.1, 1: 1632.4. Samples: 14811102. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-12 21:11:06,444][43579] Avg episode reward: [(0, '266.160'), (1, '265.010')] [2023-10-12 21:11:07,020][44959] Updated weights for policy 1, policy_version 29000 (0.0009) [2023-10-12 21:11:07,057][44958] Updated weights for policy 0, policy_version 28840 (0.0009) [2023-10-12 21:11:07,391][44959] Updated weights for policy 1, policy_version 29010 (0.0008) [2023-10-12 21:11:07,441][44958] Updated weights for policy 0, policy_version 28850 (0.0009) [2023-10-12 21:11:07,757][44959] Updated weights for policy 1, policy_version 29020 (0.0008) [2023-10-12 21:11:07,811][44958] Updated weights for policy 0, policy_version 28860 (0.0009) [2023-10-12 21:11:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59277312. Throughput: 0: 1645.8, 1: 1630.3. Samples: 14831322. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-12 21:11:11,443][43579] Avg episode reward: [(0, '264.080'), (1, '267.090')] [2023-10-12 21:11:12,017][44959] Updated weights for policy 1, policy_version 29030 (0.0007) [2023-10-12 21:11:12,073][44958] Updated weights for policy 0, policy_version 28870 (0.0009) [2023-10-12 21:11:12,400][44959] Updated weights for policy 1, policy_version 29040 (0.0009) [2023-10-12 21:11:12,436][44958] Updated weights for policy 0, policy_version 28880 (0.0009) [2023-10-12 21:11:12,764][44959] Updated weights for policy 1, policy_version 29050 (0.0007) [2023-10-12 21:11:12,802][44958] Updated weights for policy 0, policy_version 28890 (0.0008) [2023-10-12 21:11:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59342848. Throughput: 0: 1643.3, 1: 1640.4. Samples: 14851574. Policy #0 lag: (min: 29.0, avg: 53.6, max: 56.0) [2023-10-12 21:11:16,443][43579] Avg episode reward: [(0, '267.350'), (1, '271.080')] [2023-10-12 21:11:16,880][44959] Updated weights for policy 1, policy_version 29060 (0.0008) [2023-10-12 21:11:17,084][44958] Updated weights for policy 0, policy_version 28900 (0.0008) [2023-10-12 21:11:17,252][44959] Updated weights for policy 1, policy_version 29070 (0.0007) [2023-10-12 21:11:17,455][44958] Updated weights for policy 0, policy_version 28910 (0.0009) [2023-10-12 21:11:17,619][44959] Updated weights for policy 1, policy_version 29080 (0.0008) [2023-10-12 21:11:17,821][44958] Updated weights for policy 0, policy_version 28920 (0.0009) [2023-10-12 21:11:21,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59408384. Throughput: 0: 1641.4, 1: 1637.6. Samples: 14860248. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:11:21,444][43579] Avg episode reward: [(0, '268.310'), (1, '271.940')] [2023-10-12 21:11:21,744][44959] Updated weights for policy 1, policy_version 29090 (0.0008) [2023-10-12 21:11:21,995][44958] Updated weights for policy 0, policy_version 28930 (0.0009) [2023-10-12 21:11:22,106][44959] Updated weights for policy 1, policy_version 29100 (0.0007) [2023-10-12 21:11:22,370][44958] Updated weights for policy 0, policy_version 28940 (0.0007) [2023-10-12 21:11:22,478][44959] Updated weights for policy 1, policy_version 29110 (0.0007) [2023-10-12 21:11:22,747][44958] Updated weights for policy 0, policy_version 28950 (0.0009) [2023-10-12 21:11:22,850][44959] Updated weights for policy 1, policy_version 29120 (0.0008) [2023-10-12 21:11:23,125][44958] Updated weights for policy 0, policy_version 28960 (0.0007) [2023-10-12 21:11:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59473920. Throughput: 0: 1645.0, 1: 1639.0. Samples: 14880628. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:11:26,443][43579] Avg episode reward: [(0, '270.600'), (1, '274.120')] [2023-10-12 21:11:27,036][44959] Updated weights for policy 1, policy_version 29130 (0.0007) [2023-10-12 21:11:27,234][44958] Updated weights for policy 0, policy_version 28970 (0.0009) [2023-10-12 21:11:27,396][44959] Updated weights for policy 1, policy_version 29140 (0.0008) [2023-10-12 21:11:27,601][44958] Updated weights for policy 0, policy_version 28980 (0.0007) [2023-10-12 21:11:27,769][44959] Updated weights for policy 1, policy_version 29150 (0.0010) [2023-10-12 21:11:27,977][44958] Updated weights for policy 0, policy_version 28990 (0.0007) [2023-10-12 21:11:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59539456. Throughput: 0: 1648.6, 1: 1641.5. Samples: 14900728. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:11:31,443][43579] Avg episode reward: [(0, '271.890'), (1, '273.390')] [2023-10-12 21:11:31,843][44959] Updated weights for policy 1, policy_version 29160 (0.0009) [2023-10-12 21:11:32,080][44958] Updated weights for policy 0, policy_version 29000 (0.0009) [2023-10-12 21:11:32,209][44959] Updated weights for policy 1, policy_version 29170 (0.0010) [2023-10-12 21:11:32,450][44958] Updated weights for policy 0, policy_version 29010 (0.0009) [2023-10-12 21:11:32,589][44959] Updated weights for policy 1, policy_version 29180 (0.0008) [2023-10-12 21:11:32,821][44958] Updated weights for policy 0, policy_version 29020 (0.0007) [2023-10-12 21:11:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 59604992. Throughput: 0: 1650.8, 1: 1643.5. Samples: 14909708. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:11:36,443][43579] Avg episode reward: [(0, '274.930'), (1, '271.390')] [2023-10-12 21:11:37,032][44959] Updated weights for policy 1, policy_version 29190 (0.0008) [2023-10-12 21:11:37,040][44958] Updated weights for policy 0, policy_version 29030 (0.0008) [2023-10-12 21:11:37,399][44959] Updated weights for policy 1, policy_version 29200 (0.0009) [2023-10-12 21:11:37,420][44958] Updated weights for policy 0, policy_version 29040 (0.0009) [2023-10-12 21:11:37,770][44959] Updated weights for policy 1, policy_version 29210 (0.0008) [2023-10-12 21:11:37,800][44958] Updated weights for policy 0, policy_version 29050 (0.0009) [2023-10-12 21:11:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59670528. Throughput: 0: 1643.7, 1: 1642.1. Samples: 14929614. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:11:41,443][43579] Avg episode reward: [(0, '278.080'), (1, '272.640')] [2023-10-12 21:11:41,929][44958] Updated weights for policy 0, policy_version 29060 (0.0008) [2023-10-12 21:11:41,941][44959] Updated weights for policy 1, policy_version 29220 (0.0008) [2023-10-12 21:11:42,300][44958] Updated weights for policy 0, policy_version 29070 (0.0007) [2023-10-12 21:11:42,302][44959] Updated weights for policy 1, policy_version 29230 (0.0007) [2023-10-12 21:11:42,664][44958] Updated weights for policy 0, policy_version 29080 (0.0007) [2023-10-12 21:11:42,670][44959] Updated weights for policy 1, policy_version 29240 (0.0010) [2023-10-12 21:11:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59736064. Throughput: 0: 1638.3, 1: 1644.8. Samples: 14949752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:11:46,444][43579] Avg episode reward: [(0, '279.950'), (1, '268.090')] [2023-10-12 21:11:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000029088_29786112.pth... [2023-10-12 21:11:46,456][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000029248_29949952.pth... [2023-10-12 21:11:46,491][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000027712_28377088.pth [2023-10-12 21:11:46,494][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000027552_28213248.pth [2023-10-12 21:11:46,848][44959] Updated weights for policy 1, policy_version 29250 (0.0008) [2023-10-12 21:11:47,075][44958] Updated weights for policy 0, policy_version 29090 (0.0008) [2023-10-12 21:11:47,202][44959] Updated weights for policy 1, policy_version 29260 (0.0008) [2023-10-12 21:11:47,437][44958] Updated weights for policy 0, policy_version 29100 (0.0008) [2023-10-12 21:11:47,570][44959] Updated weights for policy 1, policy_version 29270 (0.0008) [2023-10-12 21:11:47,814][44958] Updated weights for policy 0, policy_version 29110 (0.0007) [2023-10-12 21:11:47,942][44959] Updated weights for policy 1, policy_version 29280 (0.0007) [2023-10-12 21:11:48,185][44958] Updated weights for policy 0, policy_version 29120 (0.0009) [2023-10-12 21:11:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59801600. Throughput: 0: 1633.4, 1: 1639.2. Samples: 14958370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:11:51,444][43579] Avg episode reward: [(0, '276.580'), (1, '265.540')] [2023-10-12 21:11:52,156][44959] Updated weights for policy 1, policy_version 29290 (0.0009) [2023-10-12 21:11:52,414][44958] Updated weights for policy 0, policy_version 29130 (0.0008) [2023-10-12 21:11:52,523][44959] Updated weights for policy 1, policy_version 29300 (0.0008) [2023-10-12 21:11:52,784][44958] Updated weights for policy 0, policy_version 29140 (0.0007) [2023-10-12 21:11:52,892][44959] Updated weights for policy 1, policy_version 29310 (0.0007) [2023-10-12 21:11:53,162][44958] Updated weights for policy 0, policy_version 29150 (0.0007) [2023-10-12 21:11:56,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59867136. Throughput: 0: 1628.4, 1: 1636.7. Samples: 14978254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:11:56,443][43579] Avg episode reward: [(0, '274.390'), (1, '264.650')] [2023-10-12 21:11:57,270][44959] Updated weights for policy 1, policy_version 29320 (0.0007) [2023-10-12 21:11:57,340][44958] Updated weights for policy 0, policy_version 29160 (0.0008) [2023-10-12 21:11:57,652][44959] Updated weights for policy 1, policy_version 29330 (0.0007) [2023-10-12 21:11:57,703][44958] Updated weights for policy 0, policy_version 29170 (0.0008) [2023-10-12 21:11:58,026][44959] Updated weights for policy 1, policy_version 29340 (0.0007) [2023-10-12 21:11:58,085][44958] Updated weights for policy 0, policy_version 29180 (0.0008) [2023-10-12 21:12:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59932672. Throughput: 0: 1624.5, 1: 1633.1. Samples: 14998166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:12:01,444][43579] Avg episode reward: [(0, '275.800'), (1, '263.600')] [2023-10-12 21:12:02,087][44959] Updated weights for policy 1, policy_version 29350 (0.0009) [2023-10-12 21:12:02,387][44958] Updated weights for policy 0, policy_version 29190 (0.0008) [2023-10-12 21:12:02,450][44959] Updated weights for policy 1, policy_version 29360 (0.0007) [2023-10-12 21:12:02,763][44958] Updated weights for policy 0, policy_version 29200 (0.0009) [2023-10-12 21:12:02,815][44959] Updated weights for policy 1, policy_version 29370 (0.0009) [2023-10-12 21:12:03,148][44958] Updated weights for policy 0, policy_version 29210 (0.0008) [2023-10-12 21:12:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 59998208. Throughput: 0: 1624.5, 1: 1633.9. Samples: 15006874. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 21:12:06,444][43579] Avg episode reward: [(0, '273.870'), (1, '258.210')] [2023-10-12 21:12:06,988][44959] Updated weights for policy 1, policy_version 29380 (0.0010) [2023-10-12 21:12:07,364][44959] Updated weights for policy 1, policy_version 29390 (0.0009) [2023-10-12 21:12:07,468][44958] Updated weights for policy 0, policy_version 29220 (0.0010) [2023-10-12 21:12:07,727][44959] Updated weights for policy 1, policy_version 29400 (0.0009) [2023-10-12 21:12:07,833][44958] Updated weights for policy 0, policy_version 29230 (0.0007) [2023-10-12 21:12:08,204][44958] Updated weights for policy 0, policy_version 29240 (0.0009) [2023-10-12 21:12:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60063744. Throughput: 0: 1619.7, 1: 1636.9. Samples: 15027176. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 21:12:11,443][43579] Avg episode reward: [(0, '272.820'), (1, '257.120')] [2023-10-12 21:12:11,733][44959] Updated weights for policy 1, policy_version 29410 (0.0007) [2023-10-12 21:12:12,098][44959] Updated weights for policy 1, policy_version 29420 (0.0008) [2023-10-12 21:12:12,288][44958] Updated weights for policy 0, policy_version 29250 (0.0008) [2023-10-12 21:12:12,463][44959] Updated weights for policy 1, policy_version 29430 (0.0007) [2023-10-12 21:12:12,664][44958] Updated weights for policy 0, policy_version 29260 (0.0007) [2023-10-12 21:12:12,830][44959] Updated weights for policy 1, policy_version 29440 (0.0007) [2023-10-12 21:12:13,037][44958] Updated weights for policy 0, policy_version 29270 (0.0007) [2023-10-12 21:12:13,413][44958] Updated weights for policy 0, policy_version 29280 (0.0010) [2023-10-12 21:12:16,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60129280. Throughput: 0: 1618.3, 1: 1642.5. Samples: 15047462. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 21:12:16,443][43579] Avg episode reward: [(0, '273.830'), (1, '261.550')] [2023-10-12 21:12:16,994][44959] Updated weights for policy 1, policy_version 29450 (0.0011) [2023-10-12 21:12:17,358][44959] Updated weights for policy 1, policy_version 29460 (0.0008) [2023-10-12 21:12:17,719][44959] Updated weights for policy 1, policy_version 29470 (0.0009) [2023-10-12 21:12:17,776][44958] Updated weights for policy 0, policy_version 29290 (0.0007) [2023-10-12 21:12:18,146][44958] Updated weights for policy 0, policy_version 29300 (0.0008) [2023-10-12 21:12:18,528][44958] Updated weights for policy 0, policy_version 29310 (0.0009) [2023-10-12 21:12:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60194816. Throughput: 0: 1617.7, 1: 1639.2. Samples: 15056270. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 21:12:21,443][43579] Avg episode reward: [(0, '275.900'), (1, '267.080')] [2023-10-12 21:12:21,944][44959] Updated weights for policy 1, policy_version 29480 (0.0010) [2023-10-12 21:12:22,319][44959] Updated weights for policy 1, policy_version 29490 (0.0010) [2023-10-12 21:12:22,664][44958] Updated weights for policy 0, policy_version 29320 (0.0007) [2023-10-12 21:12:22,685][44959] Updated weights for policy 1, policy_version 29500 (0.0008) [2023-10-12 21:12:23,036][44958] Updated weights for policy 0, policy_version 29330 (0.0008) [2023-10-12 21:12:23,410][44958] Updated weights for policy 0, policy_version 29340 (0.0007) [2023-10-12 21:12:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60260352. Throughput: 0: 1621.5, 1: 1643.9. Samples: 15076556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 21:12:26,443][43579] Avg episode reward: [(0, '271.600'), (1, '266.640')] [2023-10-12 21:12:26,741][44959] Updated weights for policy 1, policy_version 29510 (0.0008) [2023-10-12 21:12:27,107][44959] Updated weights for policy 1, policy_version 29520 (0.0009) [2023-10-12 21:12:27,482][44959] Updated weights for policy 1, policy_version 29530 (0.0009) [2023-10-12 21:12:27,529][44958] Updated weights for policy 0, policy_version 29350 (0.0008) [2023-10-12 21:12:27,905][44958] Updated weights for policy 0, policy_version 29360 (0.0010) [2023-10-12 21:12:28,288][44958] Updated weights for policy 0, policy_version 29370 (0.0010) [2023-10-12 21:12:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60325888. Throughput: 0: 1626.8, 1: 1645.4. Samples: 15097002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 21:12:31,444][43579] Avg episode reward: [(0, '272.070'), (1, '259.140')] [2023-10-12 21:12:31,567][44959] Updated weights for policy 1, policy_version 29540 (0.0007) [2023-10-12 21:12:31,932][44959] Updated weights for policy 1, policy_version 29550 (0.0009) [2023-10-12 21:12:32,300][44959] Updated weights for policy 1, policy_version 29560 (0.0009) [2023-10-12 21:12:32,389][44958] Updated weights for policy 0, policy_version 29380 (0.0010) [2023-10-12 21:12:32,754][44958] Updated weights for policy 0, policy_version 29390 (0.0010) [2023-10-12 21:12:33,128][44958] Updated weights for policy 0, policy_version 29400 (0.0010) [2023-10-12 21:12:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60391424. Throughput: 0: 1630.4, 1: 1644.9. Samples: 15105760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 21:12:36,444][43579] Avg episode reward: [(0, '273.200'), (1, '268.340')] [2023-10-12 21:12:36,628][44959] Updated weights for policy 1, policy_version 29570 (0.0008) [2023-10-12 21:12:37,002][44959] Updated weights for policy 1, policy_version 29580 (0.0008) [2023-10-12 21:12:37,180][44958] Updated weights for policy 0, policy_version 29410 (0.0010) [2023-10-12 21:12:37,362][44959] Updated weights for policy 1, policy_version 29590 (0.0010) [2023-10-12 21:12:37,551][44958] Updated weights for policy 0, policy_version 29420 (0.0007) [2023-10-12 21:12:37,725][44959] Updated weights for policy 1, policy_version 29600 (0.0009) [2023-10-12 21:12:37,920][44958] Updated weights for policy 0, policy_version 29430 (0.0009) [2023-10-12 21:12:38,284][44958] Updated weights for policy 0, policy_version 29440 (0.0011) [2023-10-12 21:12:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60456960. Throughput: 0: 1631.6, 1: 1645.9. Samples: 15125742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 21:12:41,443][43579] Avg episode reward: [(0, '270.350'), (1, '258.360')] [2023-10-12 21:12:42,020][44959] Updated weights for policy 1, policy_version 29610 (0.0010) [2023-10-12 21:12:42,388][44958] Updated weights for policy 0, policy_version 29450 (0.0009) [2023-10-12 21:12:42,390][44959] Updated weights for policy 1, policy_version 29620 (0.0008) [2023-10-12 21:12:42,761][44959] Updated weights for policy 1, policy_version 29630 (0.0008) [2023-10-12 21:12:42,768][44958] Updated weights for policy 0, policy_version 29460 (0.0007) [2023-10-12 21:12:43,144][44958] Updated weights for policy 0, policy_version 29470 (0.0008) [2023-10-12 21:12:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60522496. Throughput: 0: 1637.7, 1: 1644.9. Samples: 15145884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 21:12:46,443][43579] Avg episode reward: [(0, '271.820'), (1, '257.220')] [2023-10-12 21:12:46,871][44959] Updated weights for policy 1, policy_version 29640 (0.0010) [2023-10-12 21:12:47,237][44959] Updated weights for policy 1, policy_version 29650 (0.0010) [2023-10-12 21:12:47,478][44958] Updated weights for policy 0, policy_version 29480 (0.0009) [2023-10-12 21:12:47,606][44959] Updated weights for policy 1, policy_version 29660 (0.0007) [2023-10-12 21:12:47,847][44958] Updated weights for policy 0, policy_version 29490 (0.0009) [2023-10-12 21:12:48,230][44958] Updated weights for policy 0, policy_version 29500 (0.0009) [2023-10-12 21:12:51,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60588032. Throughput: 0: 1638.8, 1: 1644.4. Samples: 15154620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:12:51,444][43579] Avg episode reward: [(0, '273.130'), (1, '256.000')] [2023-10-12 21:12:51,769][44959] Updated weights for policy 1, policy_version 29670 (0.0008) [2023-10-12 21:12:52,146][44959] Updated weights for policy 1, policy_version 29680 (0.0008) [2023-10-12 21:12:52,339][44958] Updated weights for policy 0, policy_version 29510 (0.0010) [2023-10-12 21:12:52,506][44959] Updated weights for policy 1, policy_version 29690 (0.0008) [2023-10-12 21:12:52,703][44958] Updated weights for policy 0, policy_version 29520 (0.0008) [2023-10-12 21:12:53,077][44958] Updated weights for policy 0, policy_version 29530 (0.0010) [2023-10-12 21:12:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60653568. Throughput: 0: 1642.9, 1: 1649.1. Samples: 15175318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:12:56,444][43579] Avg episode reward: [(0, '271.690'), (1, '257.930')] [2023-10-12 21:12:56,781][44959] Updated weights for policy 1, policy_version 29700 (0.0009) [2023-10-12 21:12:57,137][44959] Updated weights for policy 1, policy_version 29710 (0.0009) [2023-10-12 21:12:57,271][44958] Updated weights for policy 0, policy_version 29540 (0.0008) [2023-10-12 21:12:57,511][44959] Updated weights for policy 1, policy_version 29720 (0.0009) [2023-10-12 21:12:57,643][44958] Updated weights for policy 0, policy_version 29550 (0.0007) [2023-10-12 21:12:58,011][44958] Updated weights for policy 0, policy_version 29560 (0.0008) [2023-10-12 21:13:01,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60719104. Throughput: 0: 1645.9, 1: 1645.7. Samples: 15195586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:13:01,443][43579] Avg episode reward: [(0, '266.950'), (1, '261.110')] [2023-10-12 21:13:01,544][44959] Updated weights for policy 1, policy_version 29730 (0.0009) [2023-10-12 21:13:01,923][44959] Updated weights for policy 1, policy_version 29740 (0.0008) [2023-10-12 21:13:02,160][44958] Updated weights for policy 0, policy_version 29570 (0.0009) [2023-10-12 21:13:02,288][44959] Updated weights for policy 1, policy_version 29750 (0.0007) [2023-10-12 21:13:02,528][44958] Updated weights for policy 0, policy_version 29580 (0.0008) [2023-10-12 21:13:02,650][44959] Updated weights for policy 1, policy_version 29760 (0.0008) [2023-10-12 21:13:02,908][44958] Updated weights for policy 0, policy_version 29590 (0.0008) [2023-10-12 21:13:03,271][44958] Updated weights for policy 0, policy_version 29600 (0.0010) [2023-10-12 21:13:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.3). Total num frames: 60784640. Throughput: 0: 1648.4, 1: 1650.5. Samples: 15204722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:13:06,443][43579] Avg episode reward: [(0, '267.130'), (1, '257.190')] [2023-10-12 21:13:06,776][44959] Updated weights for policy 1, policy_version 29770 (0.0008) [2023-10-12 21:13:07,151][44959] Updated weights for policy 1, policy_version 29780 (0.0008) [2023-10-12 21:13:07,517][44959] Updated weights for policy 1, policy_version 29790 (0.0007) [2023-10-12 21:13:07,538][44958] Updated weights for policy 0, policy_version 29610 (0.0008) [2023-10-12 21:13:07,903][44958] Updated weights for policy 0, policy_version 29620 (0.0007) [2023-10-12 21:13:08,282][44958] Updated weights for policy 0, policy_version 29630 (0.0008) [2023-10-12 21:13:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60850176. Throughput: 0: 1646.0, 1: 1647.9. Samples: 15224782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:13:11,443][43579] Avg episode reward: [(0, '265.490'), (1, '254.140')] [2023-10-12 21:13:11,637][44959] Updated weights for policy 1, policy_version 29800 (0.0010) [2023-10-12 21:13:12,013][44959] Updated weights for policy 1, policy_version 29810 (0.0010) [2023-10-12 21:13:12,266][44958] Updated weights for policy 0, policy_version 29640 (0.0009) [2023-10-12 21:13:12,391][44959] Updated weights for policy 1, policy_version 29820 (0.0008) [2023-10-12 21:13:12,633][44958] Updated weights for policy 0, policy_version 29650 (0.0008) [2023-10-12 21:13:13,009][44958] Updated weights for policy 0, policy_version 29660 (0.0007) [2023-10-12 21:13:16,260][44959] Updated weights for policy 1, policy_version 29830 (0.0010) [2023-10-12 21:13:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60915712. Throughput: 0: 1645.3, 1: 1656.4. Samples: 15245582. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:13:16,443][43579] Avg episode reward: [(0, '262.980'), (1, '250.260')] [2023-10-12 21:13:16,632][44959] Updated weights for policy 1, policy_version 29840 (0.0010) [2023-10-12 21:13:16,997][44959] Updated weights for policy 1, policy_version 29850 (0.0009) [2023-10-12 21:13:17,349][44958] Updated weights for policy 0, policy_version 29670 (0.0008) [2023-10-12 21:13:17,709][44958] Updated weights for policy 0, policy_version 29680 (0.0008) [2023-10-12 21:13:18,091][44958] Updated weights for policy 0, policy_version 29690 (0.0007) [2023-10-12 21:13:21,193][44959] Updated weights for policy 1, policy_version 29860 (0.0008) [2023-10-12 21:13:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 60981248. Throughput: 0: 1644.3, 1: 1660.0. Samples: 15254454. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:13:21,443][43579] Avg episode reward: [(0, '266.490'), (1, '249.760')] [2023-10-12 21:13:21,560][44959] Updated weights for policy 1, policy_version 29870 (0.0009) [2023-10-12 21:13:21,935][44959] Updated weights for policy 1, policy_version 29880 (0.0008) [2023-10-12 21:13:22,183][44958] Updated weights for policy 0, policy_version 29700 (0.0007) [2023-10-12 21:13:22,553][44958] Updated weights for policy 0, policy_version 29710 (0.0008) [2023-10-12 21:13:22,926][44958] Updated weights for policy 0, policy_version 29720 (0.0008) [2023-10-12 21:13:26,090][44959] Updated weights for policy 1, policy_version 29890 (0.0009) [2023-10-12 21:13:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61046784. Throughput: 0: 1648.2, 1: 1664.8. Samples: 15274828. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:13:26,443][43579] Avg episode reward: [(0, '269.770'), (1, '250.360')] [2023-10-12 21:13:26,462][44959] Updated weights for policy 1, policy_version 29900 (0.0009) [2023-10-12 21:13:26,821][44959] Updated weights for policy 1, policy_version 29910 (0.0007) [2023-10-12 21:13:27,155][44958] Updated weights for policy 0, policy_version 29730 (0.0008) [2023-10-12 21:13:27,195][44959] Updated weights for policy 1, policy_version 29920 (0.0009) [2023-10-12 21:13:27,521][44958] Updated weights for policy 0, policy_version 29740 (0.0007) [2023-10-12 21:13:27,904][44958] Updated weights for policy 0, policy_version 29750 (0.0010) [2023-10-12 21:13:28,277][44958] Updated weights for policy 0, policy_version 29760 (0.0011) [2023-10-12 21:13:31,388][44959] Updated weights for policy 1, policy_version 29930 (0.0010) [2023-10-12 21:13:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61112320. Throughput: 0: 1645.6, 1: 1662.4. Samples: 15294744. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:13:31,443][43579] Avg episode reward: [(0, '273.560'), (1, '256.060')] [2023-10-12 21:13:31,752][44959] Updated weights for policy 1, policy_version 29940 (0.0008) [2023-10-12 21:13:32,122][44959] Updated weights for policy 1, policy_version 29950 (0.0010) [2023-10-12 21:13:32,507][44958] Updated weights for policy 0, policy_version 29770 (0.0009) [2023-10-12 21:13:32,872][44958] Updated weights for policy 0, policy_version 29780 (0.0007) [2023-10-12 21:13:33,250][44958] Updated weights for policy 0, policy_version 29790 (0.0007) [2023-10-12 21:13:35,964][44959] Updated weights for policy 1, policy_version 29960 (0.0008) [2023-10-12 21:13:36,335][44959] Updated weights for policy 1, policy_version 29970 (0.0010) [2023-10-12 21:13:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61177856. Throughput: 0: 1645.4, 1: 1668.5. Samples: 15303746. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:13:36,443][43579] Avg episode reward: [(0, '273.430'), (1, '256.150')] [2023-10-12 21:13:36,705][44959] Updated weights for policy 1, policy_version 29980 (0.0008) [2023-10-12 21:13:37,415][44958] Updated weights for policy 0, policy_version 29800 (0.0008) [2023-10-12 21:13:37,796][44958] Updated weights for policy 0, policy_version 29810 (0.0008) [2023-10-12 21:13:38,172][44958] Updated weights for policy 0, policy_version 29820 (0.0007) [2023-10-12 21:13:40,698][44959] Updated weights for policy 1, policy_version 29990 (0.0007) [2023-10-12 21:13:41,060][44959] Updated weights for policy 1, policy_version 30000 (0.0010) [2023-10-12 21:13:41,430][44959] Updated weights for policy 1, policy_version 30010 (0.0007) [2023-10-12 21:13:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61243392. Throughput: 0: 1643.1, 1: 1670.9. Samples: 15324446. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-10-12 21:13:41,443][43579] Avg episode reward: [(0, '274.590'), (1, '264.000')] [2023-10-12 21:13:42,265][44958] Updated weights for policy 0, policy_version 29830 (0.0009) [2023-10-12 21:13:42,625][44958] Updated weights for policy 0, policy_version 29840 (0.0010) [2023-10-12 21:13:43,008][44958] Updated weights for policy 0, policy_version 29850 (0.0010) [2023-10-12 21:13:45,826][44959] Updated weights for policy 1, policy_version 30020 (0.0008) [2023-10-12 21:13:46,203][44959] Updated weights for policy 1, policy_version 30030 (0.0011) [2023-10-12 21:13:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61308928. Throughput: 0: 1639.7, 1: 1662.4. Samples: 15344180. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-10-12 21:13:46,443][43579] Avg episode reward: [(0, '273.370'), (1, '266.960')] [2023-10-12 21:13:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000029856_30572544.pth... [2023-10-12 21:13:46,489][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000028320_28999680.pth [2023-10-12 21:13:46,566][44959] Updated weights for policy 1, policy_version 30040 (0.0008) [2023-10-12 21:13:46,862][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000030048_30769152.pth... [2023-10-12 21:13:46,900][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000028480_29163520.pth [2023-10-12 21:13:47,210][44958] Updated weights for policy 0, policy_version 29860 (0.0010) [2023-10-12 21:13:47,586][44958] Updated weights for policy 0, policy_version 29870 (0.0007) [2023-10-12 21:13:47,953][44958] Updated weights for policy 0, policy_version 29880 (0.0007) [2023-10-12 21:13:50,492][44959] Updated weights for policy 1, policy_version 30050 (0.0008) [2023-10-12 21:13:50,865][44959] Updated weights for policy 1, policy_version 30060 (0.0008) [2023-10-12 21:13:51,233][44959] Updated weights for policy 1, policy_version 30070 (0.0009) [2023-10-12 21:13:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 61374464. Throughput: 0: 1637.6, 1: 1667.2. Samples: 15353438. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-10-12 21:13:51,443][43579] Avg episode reward: [(0, '276.120'), (1, '270.090')] [2023-10-12 21:13:51,595][44959] Updated weights for policy 1, policy_version 30080 (0.0008) [2023-10-12 21:13:52,100][44958] Updated weights for policy 0, policy_version 29890 (0.0010) [2023-10-12 21:13:52,469][44958] Updated weights for policy 0, policy_version 29900 (0.0010) [2023-10-12 21:13:52,835][44958] Updated weights for policy 0, policy_version 29910 (0.0008) [2023-10-12 21:13:53,213][44958] Updated weights for policy 0, policy_version 29920 (0.0009) [2023-10-12 21:13:55,941][44959] Updated weights for policy 1, policy_version 30090 (0.0007) [2023-10-12 21:13:56,310][44959] Updated weights for policy 1, policy_version 30100 (0.0007) [2023-10-12 21:13:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61440000. Throughput: 0: 1636.1, 1: 1668.2. Samples: 15373476. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-10-12 21:13:56,444][43579] Avg episode reward: [(0, '272.550'), (1, '274.080')] [2023-10-12 21:13:56,685][44959] Updated weights for policy 1, policy_version 30110 (0.0008) [2023-10-12 21:13:57,346][44958] Updated weights for policy 0, policy_version 29930 (0.0010) [2023-10-12 21:13:57,708][44958] Updated weights for policy 0, policy_version 29940 (0.0010) [2023-10-12 21:13:58,086][44958] Updated weights for policy 0, policy_version 29950 (0.0011) [2023-10-12 21:14:00,875][44959] Updated weights for policy 1, policy_version 30120 (0.0008) [2023-10-12 21:14:01,240][44959] Updated weights for policy 1, policy_version 30130 (0.0007) [2023-10-12 21:14:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61505536. Throughput: 0: 1628.7, 1: 1646.3. Samples: 15392956. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) [2023-10-12 21:14:01,444][43579] Avg episode reward: [(0, '273.350'), (1, '270.090')] [2023-10-12 21:14:01,606][44959] Updated weights for policy 1, policy_version 30140 (0.0007) [2023-10-12 21:14:02,463][44958] Updated weights for policy 0, policy_version 29960 (0.0009) [2023-10-12 21:14:02,831][44958] Updated weights for policy 0, policy_version 29970 (0.0009) [2023-10-12 21:14:03,210][44958] Updated weights for policy 0, policy_version 29980 (0.0009) [2023-10-12 21:14:05,693][44959] Updated weights for policy 1, policy_version 30150 (0.0008) [2023-10-12 21:14:06,067][44959] Updated weights for policy 1, policy_version 30160 (0.0009) [2023-10-12 21:14:06,424][44959] Updated weights for policy 1, policy_version 30170 (0.0008) [2023-10-12 21:14:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61571072. Throughput: 0: 1630.1, 1: 1657.2. Samples: 15402384. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) [2023-10-12 21:14:06,443][43579] Avg episode reward: [(0, '271.470'), (1, '275.070')] [2023-10-12 21:14:07,318][44958] Updated weights for policy 0, policy_version 29990 (0.0008) [2023-10-12 21:14:07,696][44958] Updated weights for policy 0, policy_version 30000 (0.0009) [2023-10-12 21:14:08,064][44958] Updated weights for policy 0, policy_version 30010 (0.0008) [2023-10-12 21:14:10,853][44959] Updated weights for policy 1, policy_version 30180 (0.0009) [2023-10-12 21:14:11,227][44959] Updated weights for policy 1, policy_version 30190 (0.0010) [2023-10-12 21:14:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61636608. Throughput: 0: 1628.1, 1: 1646.5. Samples: 15422186. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) [2023-10-12 21:14:11,443][43579] Avg episode reward: [(0, '269.150'), (1, '271.440')] [2023-10-12 21:14:11,599][44959] Updated weights for policy 1, policy_version 30200 (0.0008) [2023-10-12 21:14:12,281][44958] Updated weights for policy 0, policy_version 30020 (0.0009) [2023-10-12 21:14:12,666][44958] Updated weights for policy 0, policy_version 30030 (0.0009) [2023-10-12 21:14:13,035][44958] Updated weights for policy 0, policy_version 30040 (0.0007) [2023-10-12 21:14:15,974][44959] Updated weights for policy 1, policy_version 30210 (0.0008) [2023-10-12 21:14:16,400][44959] Updated weights for policy 1, policy_version 30220 (0.0010) [2023-10-12 21:14:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61702144. Throughput: 0: 1633.8, 1: 1642.7. Samples: 15442186. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) [2023-10-12 21:14:16,443][43579] Avg episode reward: [(0, '266.910'), (1, '271.050')] [2023-10-12 21:14:16,768][44959] Updated weights for policy 1, policy_version 30230 (0.0010) [2023-10-12 21:14:17,138][44959] Updated weights for policy 1, policy_version 30240 (0.0009) [2023-10-12 21:14:17,198][44958] Updated weights for policy 0, policy_version 30050 (0.0008) [2023-10-12 21:14:17,565][44958] Updated weights for policy 0, policy_version 30060 (0.0007) [2023-10-12 21:14:17,934][44958] Updated weights for policy 0, policy_version 30070 (0.0008) [2023-10-12 21:14:18,300][44958] Updated weights for policy 0, policy_version 30080 (0.0011) [2023-10-12 21:14:21,301][44959] Updated weights for policy 1, policy_version 30250 (0.0008) [2023-10-12 21:14:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61767680. Throughput: 0: 1636.5, 1: 1640.4. Samples: 15451204. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) [2023-10-12 21:14:21,444][43579] Avg episode reward: [(0, '265.290'), (1, '273.530')] [2023-10-12 21:14:21,667][44959] Updated weights for policy 1, policy_version 30260 (0.0008) [2023-10-12 21:14:22,039][44959] Updated weights for policy 1, policy_version 30270 (0.0009) [2023-10-12 21:14:22,592][44958] Updated weights for policy 0, policy_version 30090 (0.0008) [2023-10-12 21:14:22,962][44958] Updated weights for policy 0, policy_version 30100 (0.0008) [2023-10-12 21:14:23,340][44958] Updated weights for policy 0, policy_version 30110 (0.0008) [2023-10-12 21:14:26,164][44959] Updated weights for policy 1, policy_version 30280 (0.0008) [2023-10-12 21:14:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61833216. Throughput: 0: 1633.2, 1: 1629.5. Samples: 15471268. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 21:14:26,443][43579] Avg episode reward: [(0, '272.130'), (1, '274.120')] [2023-10-12 21:14:26,541][44959] Updated weights for policy 1, policy_version 30290 (0.0008) [2023-10-12 21:14:26,914][44959] Updated weights for policy 1, policy_version 30300 (0.0007) [2023-10-12 21:14:27,492][44958] Updated weights for policy 0, policy_version 30120 (0.0010) [2023-10-12 21:14:27,864][44958] Updated weights for policy 0, policy_version 30130 (0.0012) [2023-10-12 21:14:28,227][44958] Updated weights for policy 0, policy_version 30140 (0.0007) [2023-10-12 21:14:31,001][44959] Updated weights for policy 1, policy_version 30310 (0.0008) [2023-10-12 21:14:31,369][44959] Updated weights for policy 1, policy_version 30320 (0.0008) [2023-10-12 21:14:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61898752. Throughput: 0: 1630.6, 1: 1640.2. Samples: 15491368. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 21:14:31,443][43579] Avg episode reward: [(0, '272.520'), (1, '277.670')] [2023-10-12 21:14:31,737][44959] Updated weights for policy 1, policy_version 30330 (0.0008) [2023-10-12 21:14:32,470][44958] Updated weights for policy 0, policy_version 30150 (0.0009) [2023-10-12 21:14:32,847][44958] Updated weights for policy 0, policy_version 30160 (0.0009) [2023-10-12 21:14:33,222][44958] Updated weights for policy 0, policy_version 30170 (0.0008) [2023-10-12 21:14:35,834][44959] Updated weights for policy 1, policy_version 30340 (0.0008) [2023-10-12 21:14:36,201][44959] Updated weights for policy 1, policy_version 30350 (0.0010) [2023-10-12 21:14:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 61964288. Throughput: 0: 1632.8, 1: 1639.2. Samples: 15500682. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 21:14:36,444][43579] Avg episode reward: [(0, '275.260'), (1, '275.220')] [2023-10-12 21:14:36,566][44959] Updated weights for policy 1, policy_version 30360 (0.0010) [2023-10-12 21:14:37,422][44958] Updated weights for policy 0, policy_version 30180 (0.0009) [2023-10-12 21:14:37,799][44958] Updated weights for policy 0, policy_version 30190 (0.0008) [2023-10-12 21:14:38,175][44958] Updated weights for policy 0, policy_version 30200 (0.0007) [2023-10-12 21:14:40,568][44959] Updated weights for policy 1, policy_version 30370 (0.0008) [2023-10-12 21:14:40,938][44959] Updated weights for policy 1, policy_version 30380 (0.0008) [2023-10-12 21:14:41,307][44959] Updated weights for policy 1, policy_version 30390 (0.0007) [2023-10-12 21:14:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 62029824. Throughput: 0: 1634.5, 1: 1645.0. Samples: 15521052. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 21:14:41,444][43579] Avg episode reward: [(0, '279.650'), (1, '275.970')] [2023-10-12 21:14:41,670][44959] Updated weights for policy 1, policy_version 30400 (0.0008) [2023-10-12 21:14:42,260][44958] Updated weights for policy 0, policy_version 30210 (0.0007) [2023-10-12 21:14:42,643][44958] Updated weights for policy 0, policy_version 30220 (0.0007) [2023-10-12 21:14:43,010][44958] Updated weights for policy 0, policy_version 30230 (0.0009) [2023-10-12 21:14:43,384][44958] Updated weights for policy 0, policy_version 30240 (0.0009) [2023-10-12 21:14:45,861][44959] Updated weights for policy 1, policy_version 30410 (0.0008) [2023-10-12 21:14:46,235][44959] Updated weights for policy 1, policy_version 30420 (0.0009) [2023-10-12 21:14:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62095360. Throughput: 0: 1641.6, 1: 1643.2. Samples: 15540770. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-12 21:14:46,443][43579] Avg episode reward: [(0, '283.440'), (1, '273.360')] [2023-10-12 21:14:46,615][44959] Updated weights for policy 1, policy_version 30430 (0.0009) [2023-10-12 21:14:47,506][44958] Updated weights for policy 0, policy_version 30250 (0.0008) [2023-10-12 21:14:47,885][44958] Updated weights for policy 0, policy_version 30260 (0.0007) [2023-10-12 21:14:48,256][44958] Updated weights for policy 0, policy_version 30270 (0.0009) [2023-10-12 21:14:50,799][44959] Updated weights for policy 1, policy_version 30440 (0.0008) [2023-10-12 21:14:51,174][44959] Updated weights for policy 1, policy_version 30450 (0.0007) [2023-10-12 21:14:51,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62160896. Throughput: 0: 1640.0, 1: 1645.4. Samples: 15550224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:14:51,443][43579] Avg episode reward: [(0, '284.430'), (1, '273.370')] [2023-10-12 21:14:51,444][44518] Saving new best policy, reward=284.430! [2023-10-12 21:14:51,535][44959] Updated weights for policy 1, policy_version 30460 (0.0011) [2023-10-12 21:14:52,627][44958] Updated weights for policy 0, policy_version 30280 (0.0007) [2023-10-12 21:14:52,994][44958] Updated weights for policy 0, policy_version 30290 (0.0007) [2023-10-12 21:14:53,363][44958] Updated weights for policy 0, policy_version 30300 (0.0008) [2023-10-12 21:14:55,755][44959] Updated weights for policy 1, policy_version 30470 (0.0010) [2023-10-12 21:14:56,124][44959] Updated weights for policy 1, policy_version 30480 (0.0011) [2023-10-12 21:14:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62226432. Throughput: 0: 1638.0, 1: 1655.8. Samples: 15570404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:14:56,443][43579] Avg episode reward: [(0, '280.150'), (1, '272.770')] [2023-10-12 21:14:56,490][44959] Updated weights for policy 1, policy_version 30490 (0.0009) [2023-10-12 21:14:57,592][44958] Updated weights for policy 0, policy_version 30310 (0.0009) [2023-10-12 21:14:57,974][44958] Updated weights for policy 0, policy_version 30320 (0.0009) [2023-10-12 21:14:58,349][44958] Updated weights for policy 0, policy_version 30330 (0.0008) [2023-10-12 21:15:00,800][44959] Updated weights for policy 1, policy_version 30500 (0.0008) [2023-10-12 21:15:01,209][44959] Updated weights for policy 1, policy_version 30510 (0.0008) [2023-10-12 21:15:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62291968. Throughput: 0: 1634.6, 1: 1654.2. Samples: 15590184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:01,443][43579] Avg episode reward: [(0, '280.740'), (1, '279.430')] [2023-10-12 21:15:01,577][44959] Updated weights for policy 1, policy_version 30520 (0.0010) [2023-10-12 21:15:02,488][44958] Updated weights for policy 0, policy_version 30340 (0.0009) [2023-10-12 21:15:02,859][44958] Updated weights for policy 0, policy_version 30350 (0.0009) [2023-10-12 21:15:03,240][44958] Updated weights for policy 0, policy_version 30360 (0.0008) [2023-10-12 21:15:05,844][44959] Updated weights for policy 1, policy_version 30530 (0.0009) [2023-10-12 21:15:06,217][44959] Updated weights for policy 1, policy_version 30540 (0.0008) [2023-10-12 21:15:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62357504. Throughput: 0: 1637.4, 1: 1657.2. Samples: 15599462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:06,444][43579] Avg episode reward: [(0, '275.850'), (1, '279.620')] [2023-10-12 21:15:06,581][44959] Updated weights for policy 1, policy_version 30550 (0.0009) [2023-10-12 21:15:06,955][44959] Updated weights for policy 1, policy_version 30560 (0.0007) [2023-10-12 21:15:07,337][44958] Updated weights for policy 0, policy_version 30370 (0.0008) [2023-10-12 21:15:07,711][44958] Updated weights for policy 0, policy_version 30380 (0.0007) [2023-10-12 21:15:08,089][44958] Updated weights for policy 0, policy_version 30390 (0.0008) [2023-10-12 21:15:08,465][44958] Updated weights for policy 0, policy_version 30400 (0.0009) [2023-10-12 21:15:11,078][44959] Updated weights for policy 1, policy_version 30570 (0.0008) [2023-10-12 21:15:11,443][44959] Updated weights for policy 1, policy_version 30580 (0.0009) [2023-10-12 21:15:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62423040. Throughput: 0: 1639.4, 1: 1659.0. Samples: 15619696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:11,443][43579] Avg episode reward: [(0, '272.480'), (1, '276.090')] [2023-10-12 21:15:11,822][44959] Updated weights for policy 1, policy_version 30590 (0.0008) [2023-10-12 21:15:12,393][44958] Updated weights for policy 0, policy_version 30410 (0.0009) [2023-10-12 21:15:12,768][44958] Updated weights for policy 0, policy_version 30420 (0.0008) [2023-10-12 21:15:13,140][44958] Updated weights for policy 0, policy_version 30430 (0.0008) [2023-10-12 21:15:15,683][44959] Updated weights for policy 1, policy_version 30600 (0.0010) [2023-10-12 21:15:16,041][44959] Updated weights for policy 1, policy_version 30610 (0.0008) [2023-10-12 21:15:16,422][44959] Updated weights for policy 1, policy_version 30620 (0.0011) [2023-10-12 21:15:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62488576. Throughput: 0: 1648.8, 1: 1646.4. Samples: 15639656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:16,443][43579] Avg episode reward: [(0, '270.680'), (1, '273.510')] [2023-10-12 21:15:17,282][44958] Updated weights for policy 0, policy_version 30440 (0.0008) [2023-10-12 21:15:17,650][44958] Updated weights for policy 0, policy_version 30450 (0.0007) [2023-10-12 21:15:18,019][44958] Updated weights for policy 0, policy_version 30460 (0.0008) [2023-10-12 21:15:20,569][44959] Updated weights for policy 1, policy_version 30630 (0.0008) [2023-10-12 21:15:20,941][44959] Updated weights for policy 1, policy_version 30640 (0.0007) [2023-10-12 21:15:21,308][44959] Updated weights for policy 1, policy_version 30650 (0.0007) [2023-10-12 21:15:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62554112. Throughput: 0: 1648.7, 1: 1648.6. Samples: 15649062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:21,443][43579] Avg episode reward: [(0, '268.810'), (1, '269.430')] [2023-10-12 21:15:22,156][44958] Updated weights for policy 0, policy_version 30470 (0.0009) [2023-10-12 21:15:22,531][44958] Updated weights for policy 0, policy_version 30480 (0.0008) [2023-10-12 21:15:22,905][44958] Updated weights for policy 0, policy_version 30490 (0.0008) [2023-10-12 21:15:25,563][44959] Updated weights for policy 1, policy_version 30660 (0.0008) [2023-10-12 21:15:25,943][44959] Updated weights for policy 1, policy_version 30670 (0.0009) [2023-10-12 21:15:26,309][44959] Updated weights for policy 1, policy_version 30680 (0.0009) [2023-10-12 21:15:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62619648. Throughput: 0: 1651.7, 1: 1643.7. Samples: 15669346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:26,443][43579] Avg episode reward: [(0, '266.310'), (1, '269.120')] [2023-10-12 21:15:27,066][44958] Updated weights for policy 0, policy_version 30500 (0.0009) [2023-10-12 21:15:27,449][44958] Updated weights for policy 0, policy_version 30510 (0.0008) [2023-10-12 21:15:27,810][44958] Updated weights for policy 0, policy_version 30520 (0.0010) [2023-10-12 21:15:30,482][44959] Updated weights for policy 1, policy_version 30690 (0.0010) [2023-10-12 21:15:30,849][44959] Updated weights for policy 1, policy_version 30700 (0.0009) [2023-10-12 21:15:31,222][44959] Updated weights for policy 1, policy_version 30710 (0.0008) [2023-10-12 21:15:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62685184. Throughput: 0: 1649.6, 1: 1643.4. Samples: 15688956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:31,444][43579] Avg episode reward: [(0, '265.710'), (1, '266.500')] [2023-10-12 21:15:31,584][44959] Updated weights for policy 1, policy_version 30720 (0.0007) [2023-10-12 21:15:31,957][44958] Updated weights for policy 0, policy_version 30530 (0.0009) [2023-10-12 21:15:32,318][44958] Updated weights for policy 0, policy_version 30540 (0.0010) [2023-10-12 21:15:32,698][44958] Updated weights for policy 0, policy_version 30550 (0.0008) [2023-10-12 21:15:33,069][44958] Updated weights for policy 0, policy_version 30560 (0.0008) [2023-10-12 21:15:35,880][44959] Updated weights for policy 1, policy_version 30730 (0.0009) [2023-10-12 21:15:36,240][44959] Updated weights for policy 1, policy_version 30740 (0.0010) [2023-10-12 21:15:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62750720. Throughput: 0: 1647.5, 1: 1640.3. Samples: 15698174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:36,444][43579] Avg episode reward: [(0, '269.370'), (1, '268.810')] [2023-10-12 21:15:36,609][44959] Updated weights for policy 1, policy_version 30750 (0.0008) [2023-10-12 21:15:37,217][44958] Updated weights for policy 0, policy_version 30570 (0.0008) [2023-10-12 21:15:37,588][44958] Updated weights for policy 0, policy_version 30580 (0.0007) [2023-10-12 21:15:37,975][44958] Updated weights for policy 0, policy_version 30590 (0.0011) [2023-10-12 21:15:40,701][44959] Updated weights for policy 1, policy_version 30760 (0.0008) [2023-10-12 21:15:41,060][44959] Updated weights for policy 1, policy_version 30770 (0.0008) [2023-10-12 21:15:41,436][44959] Updated weights for policy 1, policy_version 30780 (0.0009) [2023-10-12 21:15:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62816256. Throughput: 0: 1655.0, 1: 1640.1. Samples: 15718684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:41,443][43579] Avg episode reward: [(0, '269.230'), (1, '267.700')] [2023-10-12 21:15:42,024][44958] Updated weights for policy 0, policy_version 30600 (0.0008) [2023-10-12 21:15:42,400][44958] Updated weights for policy 0, policy_version 30610 (0.0008) [2023-10-12 21:15:42,779][44958] Updated weights for policy 0, policy_version 30620 (0.0007) [2023-10-12 21:15:45,623][44959] Updated weights for policy 1, policy_version 30790 (0.0009) [2023-10-12 21:15:45,999][44959] Updated weights for policy 1, policy_version 30800 (0.0007) [2023-10-12 21:15:46,365][44959] Updated weights for policy 1, policy_version 30810 (0.0011) [2023-10-12 21:15:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62881792. Throughput: 0: 1660.0, 1: 1635.8. Samples: 15738496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:46,444][43579] Avg episode reward: [(0, '270.380'), (1, '274.220')] [2023-10-12 21:15:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000030624_31358976.pth... [2023-10-12 21:15:46,487][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000029088_29786112.pth [2023-10-12 21:15:46,492][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000030624_31358976.pth [2023-10-12 21:15:46,585][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000030816_31555584.pth... [2023-10-12 21:15:46,614][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000029248_29949952.pth [2023-10-12 21:15:46,617][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000030816_31555584.pth [2023-10-12 21:15:46,880][44958] Updated weights for policy 0, policy_version 30630 (0.0008) [2023-10-12 21:15:47,259][44958] Updated weights for policy 0, policy_version 30640 (0.0007) [2023-10-12 21:15:47,630][44958] Updated weights for policy 0, policy_version 30650 (0.0008) [2023-10-12 21:15:50,687][44959] Updated weights for policy 1, policy_version 30820 (0.0008) [2023-10-12 21:15:51,086][44959] Updated weights for policy 1, policy_version 30830 (0.0008) [2023-10-12 21:15:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 62947328. Throughput: 0: 1656.1, 1: 1640.0. Samples: 15747788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:51,444][44959] Updated weights for policy 1, policy_version 30840 (0.0009) [2023-10-12 21:15:51,444][43579] Avg episode reward: [(0, '272.980'), (1, '275.490')] [2023-10-12 21:15:51,685][44958] Updated weights for policy 0, policy_version 30660 (0.0007) [2023-10-12 21:15:52,057][44958] Updated weights for policy 0, policy_version 30670 (0.0010) [2023-10-12 21:15:52,427][44958] Updated weights for policy 0, policy_version 30680 (0.0007) [2023-10-12 21:15:55,463][44959] Updated weights for policy 1, policy_version 30850 (0.0009) [2023-10-12 21:15:55,834][44959] Updated weights for policy 1, policy_version 30860 (0.0009) [2023-10-12 21:15:56,208][44959] Updated weights for policy 1, policy_version 30870 (0.0008) [2023-10-12 21:15:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 63012864. Throughput: 0: 1657.0, 1: 1640.6. Samples: 15768088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:15:56,443][43579] Avg episode reward: [(0, '270.960'), (1, '272.100')] [2023-10-12 21:15:56,576][44959] Updated weights for policy 1, policy_version 30880 (0.0009) [2023-10-12 21:15:56,580][44958] Updated weights for policy 0, policy_version 30690 (0.0008) [2023-10-12 21:15:56,955][44958] Updated weights for policy 0, policy_version 30700 (0.0010) [2023-10-12 21:15:57,331][44958] Updated weights for policy 0, policy_version 30710 (0.0008) [2023-10-12 21:15:57,701][44958] Updated weights for policy 0, policy_version 30720 (0.0009) [2023-10-12 21:16:00,716][44959] Updated weights for policy 1, policy_version 30890 (0.0007) [2023-10-12 21:16:01,079][44959] Updated weights for policy 1, policy_version 30900 (0.0010) [2023-10-12 21:16:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 63078400. Throughput: 0: 1653.0, 1: 1638.4. Samples: 15787770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:16:01,443][43579] Avg episode reward: [(0, '268.740'), (1, '271.470')] [2023-10-12 21:16:01,450][44959] Updated weights for policy 1, policy_version 30910 (0.0011) [2023-10-12 21:16:01,973][44958] Updated weights for policy 0, policy_version 30730 (0.0010) [2023-10-12 21:16:02,344][44958] Updated weights for policy 0, policy_version 30740 (0.0009) [2023-10-12 21:16:02,708][44958] Updated weights for policy 0, policy_version 30750 (0.0010) [2023-10-12 21:16:05,672][44959] Updated weights for policy 1, policy_version 30920 (0.0010) [2023-10-12 21:16:06,037][44959] Updated weights for policy 1, policy_version 30930 (0.0007) [2023-10-12 21:16:06,408][44959] Updated weights for policy 1, policy_version 30940 (0.0007) [2023-10-12 21:16:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 63143936. Throughput: 0: 1650.4, 1: 1646.5. Samples: 15797426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:16:06,443][43579] Avg episode reward: [(0, '268.470'), (1, '270.500')] [2023-10-12 21:16:06,869][44958] Updated weights for policy 0, policy_version 30760 (0.0011) [2023-10-12 21:16:07,234][44958] Updated weights for policy 0, policy_version 30770 (0.0010) [2023-10-12 21:16:07,603][44958] Updated weights for policy 0, policy_version 30780 (0.0010) [2023-10-12 21:16:10,406][44959] Updated weights for policy 1, policy_version 30950 (0.0008) [2023-10-12 21:16:10,768][44959] Updated weights for policy 1, policy_version 30960 (0.0008) [2023-10-12 21:16:11,143][44959] Updated weights for policy 1, policy_version 30970 (0.0008) [2023-10-12 21:16:11,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63242240. Throughput: 0: 1653.6, 1: 1648.8. Samples: 15817954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:16:11,444][43579] Avg episode reward: [(0, '269.220'), (1, '264.190')] [2023-10-12 21:16:11,840][44958] Updated weights for policy 0, policy_version 30790 (0.0008) [2023-10-12 21:16:12,216][44958] Updated weights for policy 0, policy_version 30800 (0.0008) [2023-10-12 21:16:12,586][44958] Updated weights for policy 0, policy_version 30810 (0.0007) [2023-10-12 21:16:15,206][44959] Updated weights for policy 1, policy_version 30980 (0.0009) [2023-10-12 21:16:15,583][44959] Updated weights for policy 1, policy_version 30990 (0.0008) [2023-10-12 21:16:15,947][44959] Updated weights for policy 1, policy_version 31000 (0.0008) [2023-10-12 21:16:16,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63307776. Throughput: 0: 1654.0, 1: 1646.5. Samples: 15837478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:16:16,443][43579] Avg episode reward: [(0, '272.060'), (1, '265.690')] [2023-10-12 21:16:16,709][44958] Updated weights for policy 0, policy_version 30820 (0.0009) [2023-10-12 21:16:17,085][44958] Updated weights for policy 0, policy_version 30830 (0.0010) [2023-10-12 21:16:17,456][44958] Updated weights for policy 0, policy_version 30840 (0.0011) [2023-10-12 21:16:20,024][44959] Updated weights for policy 1, policy_version 31010 (0.0007) [2023-10-12 21:16:20,392][44959] Updated weights for policy 1, policy_version 31020 (0.0007) [2023-10-12 21:16:20,766][44959] Updated weights for policy 1, policy_version 31030 (0.0008) [2023-10-12 21:16:21,127][44959] Updated weights for policy 1, policy_version 31040 (0.0007) [2023-10-12 21:16:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63373312. Throughput: 0: 1654.3, 1: 1659.8. Samples: 15847308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:16:21,443][43579] Avg episode reward: [(0, '269.200'), (1, '262.790')] [2023-10-12 21:16:21,554][44958] Updated weights for policy 0, policy_version 30850 (0.0010) [2023-10-12 21:16:21,925][44958] Updated weights for policy 0, policy_version 30860 (0.0009) [2023-10-12 21:16:22,299][44958] Updated weights for policy 0, policy_version 30870 (0.0008) [2023-10-12 21:16:22,670][44958] Updated weights for policy 0, policy_version 30880 (0.0011) [2023-10-12 21:16:25,399][44959] Updated weights for policy 1, policy_version 31050 (0.0010) [2023-10-12 21:16:25,765][44959] Updated weights for policy 1, policy_version 31060 (0.0010) [2023-10-12 21:16:26,136][44959] Updated weights for policy 1, policy_version 31070 (0.0009) [2023-10-12 21:16:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63438848. Throughput: 0: 1653.0, 1: 1660.7. Samples: 15867802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:16:26,444][43579] Avg episode reward: [(0, '273.040'), (1, '266.770')] [2023-10-12 21:16:26,903][44958] Updated weights for policy 0, policy_version 30890 (0.0009) [2023-10-12 21:16:27,275][44958] Updated weights for policy 0, policy_version 30900 (0.0008) [2023-10-12 21:16:27,662][44958] Updated weights for policy 0, policy_version 30910 (0.0011) [2023-10-12 21:16:30,013][44959] Updated weights for policy 1, policy_version 31080 (0.0008) [2023-10-12 21:16:30,395][44959] Updated weights for policy 1, policy_version 31090 (0.0008) [2023-10-12 21:16:30,761][44959] Updated weights for policy 1, policy_version 31100 (0.0008) [2023-10-12 21:16:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63504384. Throughput: 0: 1640.5, 1: 1649.5. Samples: 15886548. Policy #0 lag: (min: 26.0, avg: 33.3, max: 58.0) [2023-10-12 21:16:31,444][43579] Avg episode reward: [(0, '272.130'), (1, '262.590')] [2023-10-12 21:16:31,835][44958] Updated weights for policy 0, policy_version 30920 (0.0010) [2023-10-12 21:16:32,213][44958] Updated weights for policy 0, policy_version 30930 (0.0010) [2023-10-12 21:16:32,586][44958] Updated weights for policy 0, policy_version 30940 (0.0009) [2023-10-12 21:16:35,064][44959] Updated weights for policy 1, policy_version 31110 (0.0008) [2023-10-12 21:16:35,463][44959] Updated weights for policy 1, policy_version 31120 (0.0008) [2023-10-12 21:16:35,835][44959] Updated weights for policy 1, policy_version 31130 (0.0008) [2023-10-12 21:16:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63569920. Throughput: 0: 1639.6, 1: 1666.7. Samples: 15896574. Policy #0 lag: (min: 26.0, avg: 33.3, max: 58.0) [2023-10-12 21:16:36,444][43579] Avg episode reward: [(0, '270.740'), (1, '272.300')] [2023-10-12 21:16:36,819][44958] Updated weights for policy 0, policy_version 30950 (0.0008) [2023-10-12 21:16:37,199][44958] Updated weights for policy 0, policy_version 30960 (0.0008) [2023-10-12 21:16:37,561][44958] Updated weights for policy 0, policy_version 30970 (0.0007) [2023-10-12 21:16:39,916][44959] Updated weights for policy 1, policy_version 31140 (0.0010) [2023-10-12 21:16:40,277][44959] Updated weights for policy 1, policy_version 31150 (0.0009) [2023-10-12 21:16:40,644][44959] Updated weights for policy 1, policy_version 31160 (0.0009) [2023-10-12 21:16:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63635456. Throughput: 0: 1644.1, 1: 1657.1. Samples: 15916642. Policy #0 lag: (min: 26.0, avg: 33.3, max: 58.0) [2023-10-12 21:16:41,444][43579] Avg episode reward: [(0, '267.060'), (1, '273.350')] [2023-10-12 21:16:41,563][44958] Updated weights for policy 0, policy_version 30980 (0.0009) [2023-10-12 21:16:41,932][44958] Updated weights for policy 0, policy_version 30990 (0.0007) [2023-10-12 21:16:42,305][44958] Updated weights for policy 0, policy_version 31000 (0.0007) [2023-10-12 21:16:44,716][44959] Updated weights for policy 1, policy_version 31170 (0.0008) [2023-10-12 21:16:45,087][44959] Updated weights for policy 1, policy_version 31180 (0.0008) [2023-10-12 21:16:45,463][44959] Updated weights for policy 1, policy_version 31190 (0.0010) [2023-10-12 21:16:45,829][44959] Updated weights for policy 1, policy_version 31200 (0.0010) [2023-10-12 21:16:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 63700992. Throughput: 0: 1647.7, 1: 1647.1. Samples: 15936036. Policy #0 lag: (min: 26.0, avg: 33.3, max: 58.0) [2023-10-12 21:16:46,443][43579] Avg episode reward: [(0, '268.350'), (1, '275.870')] [2023-10-12 21:16:46,561][44958] Updated weights for policy 0, policy_version 31010 (0.0008) [2023-10-12 21:16:46,935][44958] Updated weights for policy 0, policy_version 31020 (0.0008) [2023-10-12 21:16:47,301][44958] Updated weights for policy 0, policy_version 31030 (0.0008) [2023-10-12 21:16:47,681][44958] Updated weights for policy 0, policy_version 31040 (0.0008) [2023-10-12 21:16:50,175][44959] Updated weights for policy 1, policy_version 31210 (0.0008) [2023-10-12 21:16:50,548][44959] Updated weights for policy 1, policy_version 31220 (0.0007) [2023-10-12 21:16:50,924][44959] Updated weights for policy 1, policy_version 31230 (0.0007) [2023-10-12 21:16:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 63766528. Throughput: 0: 1650.1, 1: 1652.6. Samples: 15946046. Policy #0 lag: (min: 26.0, avg: 33.3, max: 58.0) [2023-10-12 21:16:51,443][43579] Avg episode reward: [(0, '260.940'), (1, '276.960')] [2023-10-12 21:16:51,801][44958] Updated weights for policy 0, policy_version 31050 (0.0008) [2023-10-12 21:16:52,186][44958] Updated weights for policy 0, policy_version 31060 (0.0008) [2023-10-12 21:16:52,557][44958] Updated weights for policy 0, policy_version 31070 (0.0010) [2023-10-12 21:16:55,153][44959] Updated weights for policy 1, policy_version 31240 (0.0009) [2023-10-12 21:16:55,526][44959] Updated weights for policy 1, policy_version 31250 (0.0008) [2023-10-12 21:16:55,892][44959] Updated weights for policy 1, policy_version 31260 (0.0009) [2023-10-12 21:16:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63832064. Throughput: 0: 1651.3, 1: 1648.5. Samples: 15966446. Policy #0 lag: (min: 0.0, avg: 15.2, max: 32.0) [2023-10-12 21:16:56,444][43579] Avg episode reward: [(0, '266.330'), (1, '281.550')] [2023-10-12 21:16:56,603][44958] Updated weights for policy 0, policy_version 31080 (0.0009) [2023-10-12 21:16:56,978][44958] Updated weights for policy 0, policy_version 31090 (0.0010) [2023-10-12 21:16:57,346][44958] Updated weights for policy 0, policy_version 31100 (0.0007) [2023-10-12 21:16:59,958][44959] Updated weights for policy 1, policy_version 31270 (0.0008) [2023-10-12 21:17:00,325][44959] Updated weights for policy 1, policy_version 31280 (0.0008) [2023-10-12 21:17:00,710][44959] Updated weights for policy 1, policy_version 31290 (0.0008) [2023-10-12 21:17:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63897600. Throughput: 0: 1647.2, 1: 1638.3. Samples: 15985326. Policy #0 lag: (min: 0.0, avg: 15.2, max: 32.0) [2023-10-12 21:17:01,443][43579] Avg episode reward: [(0, '267.630'), (1, '281.750')] [2023-10-12 21:17:01,744][44958] Updated weights for policy 0, policy_version 31110 (0.0009) [2023-10-12 21:17:02,115][44958] Updated weights for policy 0, policy_version 31120 (0.0009) [2023-10-12 21:17:02,499][44958] Updated weights for policy 0, policy_version 31130 (0.0010) [2023-10-12 21:17:04,845][44959] Updated weights for policy 1, policy_version 31300 (0.0008) [2023-10-12 21:17:05,213][44959] Updated weights for policy 1, policy_version 31310 (0.0008) [2023-10-12 21:17:05,572][44959] Updated weights for policy 1, policy_version 31320 (0.0007) [2023-10-12 21:17:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 63963136. Throughput: 0: 1648.2, 1: 1643.6. Samples: 15995438. Policy #0 lag: (min: 0.0, avg: 15.2, max: 32.0) [2023-10-12 21:17:06,443][43579] Avg episode reward: [(0, '272.940'), (1, '282.280')] [2023-10-12 21:17:06,496][44958] Updated weights for policy 0, policy_version 31140 (0.0008) [2023-10-12 21:17:06,873][44958] Updated weights for policy 0, policy_version 31150 (0.0008) [2023-10-12 21:17:07,251][44958] Updated weights for policy 0, policy_version 31160 (0.0009) [2023-10-12 21:17:09,761][44959] Updated weights for policy 1, policy_version 31330 (0.0009) [2023-10-12 21:17:10,133][44959] Updated weights for policy 1, policy_version 31340 (0.0009) [2023-10-12 21:17:10,516][44959] Updated weights for policy 1, policy_version 31350 (0.0009) [2023-10-12 21:17:10,882][44959] Updated weights for policy 1, policy_version 31360 (0.0009) [2023-10-12 21:17:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64028672. Throughput: 0: 1641.2, 1: 1639.2. Samples: 16015420. Policy #0 lag: (min: 0.0, avg: 15.2, max: 32.0) [2023-10-12 21:17:11,443][43579] Avg episode reward: [(0, '271.620'), (1, '279.430')] [2023-10-12 21:17:11,538][44958] Updated weights for policy 0, policy_version 31170 (0.0008) [2023-10-12 21:17:11,907][44958] Updated weights for policy 0, policy_version 31180 (0.0009) [2023-10-12 21:17:12,278][44958] Updated weights for policy 0, policy_version 31190 (0.0007) [2023-10-12 21:17:12,652][44958] Updated weights for policy 0, policy_version 31200 (0.0009) [2023-10-12 21:17:14,980][44959] Updated weights for policy 1, policy_version 31370 (0.0009) [2023-10-12 21:17:15,353][44959] Updated weights for policy 1, policy_version 31380 (0.0009) [2023-10-12 21:17:15,722][44959] Updated weights for policy 1, policy_version 31390 (0.0008) [2023-10-12 21:17:16,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 64094208. Throughput: 0: 1657.0, 1: 1643.8. Samples: 16035084. Policy #0 lag: (min: 0.0, avg: 15.2, max: 32.0) [2023-10-12 21:17:16,444][43579] Avg episode reward: [(0, '268.890'), (1, '272.930')] [2023-10-12 21:17:16,733][44958] Updated weights for policy 0, policy_version 31210 (0.0009) [2023-10-12 21:17:17,108][44958] Updated weights for policy 0, policy_version 31220 (0.0007) [2023-10-12 21:17:17,485][44958] Updated weights for policy 0, policy_version 31230 (0.0009) [2023-10-12 21:17:19,908][44959] Updated weights for policy 1, policy_version 31400 (0.0008) [2023-10-12 21:17:20,289][44959] Updated weights for policy 1, policy_version 31410 (0.0008) [2023-10-12 21:17:20,662][44959] Updated weights for policy 1, policy_version 31420 (0.0007) [2023-10-12 21:17:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64159744. Throughput: 0: 1655.2, 1: 1646.4. Samples: 16045148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:17:21,444][43579] Avg episode reward: [(0, '276.350'), (1, '267.100')] [2023-10-12 21:17:21,694][44958] Updated weights for policy 0, policy_version 31240 (0.0010) [2023-10-12 21:17:22,072][44958] Updated weights for policy 0, policy_version 31250 (0.0009) [2023-10-12 21:17:22,439][44958] Updated weights for policy 0, policy_version 31260 (0.0008) [2023-10-12 21:17:24,956][44959] Updated weights for policy 1, policy_version 31430 (0.0009) [2023-10-12 21:17:25,319][44959] Updated weights for policy 1, policy_version 31440 (0.0009) [2023-10-12 21:17:25,695][44959] Updated weights for policy 1, policy_version 31450 (0.0010) [2023-10-12 21:17:26,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64225280. Throughput: 0: 1653.2, 1: 1640.2. Samples: 16064846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:17:26,444][43579] Avg episode reward: [(0, '273.810'), (1, '270.140')] [2023-10-12 21:17:26,465][44958] Updated weights for policy 0, policy_version 31270 (0.0007) [2023-10-12 21:17:26,832][44958] Updated weights for policy 0, policy_version 31280 (0.0009) [2023-10-12 21:17:27,203][44958] Updated weights for policy 0, policy_version 31290 (0.0008) [2023-10-12 21:17:29,861][44959] Updated weights for policy 1, policy_version 31460 (0.0010) [2023-10-12 21:17:30,232][44959] Updated weights for policy 1, policy_version 31470 (0.0009) [2023-10-12 21:17:30,610][44959] Updated weights for policy 1, policy_version 31480 (0.0008) [2023-10-12 21:17:31,008][44958] Updated weights for policy 0, policy_version 31300 (0.0009) [2023-10-12 21:17:31,376][44958] Updated weights for policy 0, policy_version 31310 (0.0010) [2023-10-12 21:17:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64290816. Throughput: 0: 1652.9, 1: 1640.5. Samples: 16084240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:17:31,443][43579] Avg episode reward: [(0, '271.200'), (1, '266.030')] [2023-10-12 21:17:31,750][44958] Updated weights for policy 0, policy_version 31320 (0.0009) [2023-10-12 21:17:34,702][44959] Updated weights for policy 1, policy_version 31490 (0.0009) [2023-10-12 21:17:35,068][44959] Updated weights for policy 1, policy_version 31500 (0.0009) [2023-10-12 21:17:35,447][44959] Updated weights for policy 1, policy_version 31510 (0.0009) [2023-10-12 21:17:35,821][44959] Updated weights for policy 1, policy_version 31520 (0.0009) [2023-10-12 21:17:36,085][44958] Updated weights for policy 0, policy_version 31330 (0.0010) [2023-10-12 21:17:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64356352. Throughput: 0: 1653.9, 1: 1644.6. Samples: 16094476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:17:36,444][43579] Avg episode reward: [(0, '270.620'), (1, '261.860')] [2023-10-12 21:17:36,454][44958] Updated weights for policy 0, policy_version 31340 (0.0009) [2023-10-12 21:17:36,822][44958] Updated weights for policy 0, policy_version 31350 (0.0007) [2023-10-12 21:17:37,195][44958] Updated weights for policy 0, policy_version 31360 (0.0009) [2023-10-12 21:17:39,974][44959] Updated weights for policy 1, policy_version 31530 (0.0008) [2023-10-12 21:17:40,350][44959] Updated weights for policy 1, policy_version 31540 (0.0009) [2023-10-12 21:17:40,714][44959] Updated weights for policy 1, policy_version 31550 (0.0008) [2023-10-12 21:17:41,440][44958] Updated weights for policy 0, policy_version 31370 (0.0007) [2023-10-12 21:17:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64421888. Throughput: 0: 1645.4, 1: 1639.9. Samples: 16114282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:17:41,443][43579] Avg episode reward: [(0, '267.430'), (1, '263.420')] [2023-10-12 21:17:41,806][44958] Updated weights for policy 0, policy_version 31380 (0.0007) [2023-10-12 21:17:42,190][44958] Updated weights for policy 0, policy_version 31390 (0.0007) [2023-10-12 21:17:44,831][44959] Updated weights for policy 1, policy_version 31560 (0.0007) [2023-10-12 21:17:45,198][44959] Updated weights for policy 1, policy_version 31570 (0.0009) [2023-10-12 21:17:45,569][44959] Updated weights for policy 1, policy_version 31580 (0.0008) [2023-10-12 21:17:46,322][44958] Updated weights for policy 0, policy_version 31400 (0.0009) [2023-10-12 21:17:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64487424. Throughput: 0: 1650.2, 1: 1650.3. Samples: 16133848. Policy #0 lag: (min: 18.0, avg: 19.0, max: 38.0) [2023-10-12 21:17:46,443][43579] Avg episode reward: [(0, '264.780'), (1, '265.230')] [2023-10-12 21:17:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000031584_32342016.pth... [2023-10-12 21:17:46,481][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000030048_30769152.pth [2023-10-12 21:17:46,698][44958] Updated weights for policy 0, policy_version 31410 (0.0008) [2023-10-12 21:17:47,070][44958] Updated weights for policy 0, policy_version 31420 (0.0009) [2023-10-12 21:17:47,216][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000031424_32178176.pth... [2023-10-12 21:17:47,246][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000029856_30572544.pth [2023-10-12 21:17:49,740][44959] Updated weights for policy 1, policy_version 31590 (0.0007) [2023-10-12 21:17:50,110][44959] Updated weights for policy 1, policy_version 31600 (0.0007) [2023-10-12 21:17:50,471][44959] Updated weights for policy 1, policy_version 31610 (0.0008) [2023-10-12 21:17:51,417][44958] Updated weights for policy 0, policy_version 31430 (0.0010) [2023-10-12 21:17:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64552960. Throughput: 0: 1650.4, 1: 1648.7. Samples: 16143898. Policy #0 lag: (min: 18.0, avg: 19.0, max: 38.0) [2023-10-12 21:17:51,443][43579] Avg episode reward: [(0, '263.660'), (1, '268.540')] [2023-10-12 21:17:51,788][44958] Updated weights for policy 0, policy_version 31440 (0.0009) [2023-10-12 21:17:52,167][44958] Updated weights for policy 0, policy_version 31450 (0.0011) [2023-10-12 21:17:54,832][44959] Updated weights for policy 1, policy_version 31620 (0.0008) [2023-10-12 21:17:55,201][44959] Updated weights for policy 1, policy_version 31630 (0.0008) [2023-10-12 21:17:55,558][44959] Updated weights for policy 1, policy_version 31640 (0.0009) [2023-10-12 21:17:56,342][44958] Updated weights for policy 0, policy_version 31460 (0.0009) [2023-10-12 21:17:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64618496. Throughput: 0: 1654.4, 1: 1644.8. Samples: 16163886. Policy #0 lag: (min: 18.0, avg: 19.0, max: 38.0) [2023-10-12 21:17:56,443][43579] Avg episode reward: [(0, '261.380'), (1, '270.000')] [2023-10-12 21:17:56,709][44958] Updated weights for policy 0, policy_version 31470 (0.0009) [2023-10-12 21:17:57,083][44958] Updated weights for policy 0, policy_version 31480 (0.0008) [2023-10-12 21:17:59,569][44959] Updated weights for policy 1, policy_version 31650 (0.0009) [2023-10-12 21:17:59,941][44959] Updated weights for policy 1, policy_version 31660 (0.0009) [2023-10-12 21:18:00,309][44959] Updated weights for policy 1, policy_version 31670 (0.0008) [2023-10-12 21:18:00,678][44959] Updated weights for policy 1, policy_version 31680 (0.0007) [2023-10-12 21:18:01,009][44958] Updated weights for policy 0, policy_version 31490 (0.0008) [2023-10-12 21:18:01,386][44958] Updated weights for policy 0, policy_version 31500 (0.0009) [2023-10-12 21:18:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64684032. Throughput: 0: 1644.8, 1: 1648.8. Samples: 16183296. Policy #0 lag: (min: 18.0, avg: 19.0, max: 38.0) [2023-10-12 21:18:01,443][43579] Avg episode reward: [(0, '264.820'), (1, '270.970')] [2023-10-12 21:18:01,757][44958] Updated weights for policy 0, policy_version 31510 (0.0008) [2023-10-12 21:18:02,133][44958] Updated weights for policy 0, policy_version 31520 (0.0009) [2023-10-12 21:18:04,993][44959] Updated weights for policy 1, policy_version 31690 (0.0007) [2023-10-12 21:18:05,369][44959] Updated weights for policy 1, policy_version 31700 (0.0008) [2023-10-12 21:18:05,745][44959] Updated weights for policy 1, policy_version 31710 (0.0010) [2023-10-12 21:18:06,343][44958] Updated weights for policy 0, policy_version 31530 (0.0008) [2023-10-12 21:18:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64749568. Throughput: 0: 1649.0, 1: 1644.9. Samples: 16193370. Policy #0 lag: (min: 18.0, avg: 19.0, max: 38.0) [2023-10-12 21:18:06,443][43579] Avg episode reward: [(0, '261.750'), (1, '274.720')] [2023-10-12 21:18:06,723][44958] Updated weights for policy 0, policy_version 31540 (0.0011) [2023-10-12 21:18:07,102][44958] Updated weights for policy 0, policy_version 31550 (0.0010) [2023-10-12 21:18:09,802][44959] Updated weights for policy 1, policy_version 31720 (0.0009) [2023-10-12 21:18:10,171][44959] Updated weights for policy 1, policy_version 31730 (0.0009) [2023-10-12 21:18:10,548][44959] Updated weights for policy 1, policy_version 31740 (0.0007) [2023-10-12 21:18:11,363][44958] Updated weights for policy 0, policy_version 31560 (0.0008) [2023-10-12 21:18:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64815104. Throughput: 0: 1647.1, 1: 1646.4. Samples: 16213056. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-12 21:18:11,444][43579] Avg episode reward: [(0, '265.590'), (1, '276.060')] [2023-10-12 21:18:11,742][44958] Updated weights for policy 0, policy_version 31570 (0.0008) [2023-10-12 21:18:12,114][44958] Updated weights for policy 0, policy_version 31580 (0.0009) [2023-10-12 21:18:14,729][44959] Updated weights for policy 1, policy_version 31750 (0.0010) [2023-10-12 21:18:15,112][44959] Updated weights for policy 1, policy_version 31760 (0.0011) [2023-10-12 21:18:15,491][44959] Updated weights for policy 1, policy_version 31770 (0.0009) [2023-10-12 21:18:16,155][44958] Updated weights for policy 0, policy_version 31590 (0.0008) [2023-10-12 21:18:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 64880640. Throughput: 0: 1637.2, 1: 1650.8. Samples: 16232202. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-12 21:18:16,443][43579] Avg episode reward: [(0, '269.330'), (1, '273.520')] [2023-10-12 21:18:16,530][44958] Updated weights for policy 0, policy_version 31600 (0.0008) [2023-10-12 21:18:16,898][44958] Updated weights for policy 0, policy_version 31610 (0.0007) [2023-10-12 21:18:19,746][44959] Updated weights for policy 1, policy_version 31780 (0.0009) [2023-10-12 21:18:20,123][44959] Updated weights for policy 1, policy_version 31790 (0.0009) [2023-10-12 21:18:20,506][44959] Updated weights for policy 1, policy_version 31800 (0.0009) [2023-10-12 21:18:21,329][44958] Updated weights for policy 0, policy_version 31620 (0.0008) [2023-10-12 21:18:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64946176. Throughput: 0: 1638.9, 1: 1647.7. Samples: 16242374. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-12 21:18:21,444][43579] Avg episode reward: [(0, '268.320'), (1, '271.370')] [2023-10-12 21:18:21,702][44958] Updated weights for policy 0, policy_version 31630 (0.0010) [2023-10-12 21:18:22,066][44958] Updated weights for policy 0, policy_version 31640 (0.0008) [2023-10-12 21:18:24,639][44959] Updated weights for policy 1, policy_version 31810 (0.0008) [2023-10-12 21:18:25,011][44959] Updated weights for policy 1, policy_version 31820 (0.0008) [2023-10-12 21:18:25,370][44959] Updated weights for policy 1, policy_version 31830 (0.0009) [2023-10-12 21:18:25,741][44959] Updated weights for policy 1, policy_version 31840 (0.0011) [2023-10-12 21:18:26,062][44958] Updated weights for policy 0, policy_version 31650 (0.0007) [2023-10-12 21:18:26,441][44958] Updated weights for policy 0, policy_version 31660 (0.0009) [2023-10-12 21:18:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65011712. Throughput: 0: 1638.8, 1: 1649.6. Samples: 16262262. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-12 21:18:26,444][43579] Avg episode reward: [(0, '267.960'), (1, '271.430')] [2023-10-12 21:18:26,802][44958] Updated weights for policy 0, policy_version 31670 (0.0010) [2023-10-12 21:18:27,171][44958] Updated weights for policy 0, policy_version 31680 (0.0010) [2023-10-12 21:18:29,540][44959] Updated weights for policy 1, policy_version 31850 (0.0007) [2023-10-12 21:18:29,906][44959] Updated weights for policy 1, policy_version 31860 (0.0012) [2023-10-12 21:18:30,275][44959] Updated weights for policy 1, policy_version 31870 (0.0009) [2023-10-12 21:18:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65077248. Throughput: 0: 1634.8, 1: 1651.1. Samples: 16281710. Policy #0 lag: (min: 14.0, avg: 16.7, max: 46.0) [2023-10-12 21:18:31,443][43579] Avg episode reward: [(0, '261.530'), (1, '270.150')] [2023-10-12 21:18:31,535][44958] Updated weights for policy 0, policy_version 31690 (0.0008) [2023-10-12 21:18:31,906][44958] Updated weights for policy 0, policy_version 31700 (0.0007) [2023-10-12 21:18:32,279][44958] Updated weights for policy 0, policy_version 31710 (0.0008) [2023-10-12 21:18:34,389][44959] Updated weights for policy 1, policy_version 31880 (0.0008) [2023-10-12 21:18:34,755][44959] Updated weights for policy 1, policy_version 31890 (0.0010) [2023-10-12 21:18:35,124][44959] Updated weights for policy 1, policy_version 31900 (0.0008) [2023-10-12 21:18:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65142784. Throughput: 0: 1636.0, 1: 1651.4. Samples: 16291828. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 21:18:36,443][43579] Avg episode reward: [(0, '255.360'), (1, '269.920')] [2023-10-12 21:18:36,506][44958] Updated weights for policy 0, policy_version 31720 (0.0008) [2023-10-12 21:18:36,877][44958] Updated weights for policy 0, policy_version 31730 (0.0010) [2023-10-12 21:18:37,250][44958] Updated weights for policy 0, policy_version 31740 (0.0011) [2023-10-12 21:18:39,475][44959] Updated weights for policy 1, policy_version 31910 (0.0007) [2023-10-12 21:18:39,842][44959] Updated weights for policy 1, policy_version 31920 (0.0009) [2023-10-12 21:18:40,205][44959] Updated weights for policy 1, policy_version 31930 (0.0010) [2023-10-12 21:18:41,397][44958] Updated weights for policy 0, policy_version 31750 (0.0011) [2023-10-12 21:18:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65208320. Throughput: 0: 1636.6, 1: 1641.2. Samples: 16311388. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 21:18:41,444][43579] Avg episode reward: [(0, '254.520'), (1, '270.200')] [2023-10-12 21:18:41,775][44958] Updated weights for policy 0, policy_version 31760 (0.0009) [2023-10-12 21:18:42,144][44958] Updated weights for policy 0, policy_version 31770 (0.0007) [2023-10-12 21:18:44,202][44959] Updated weights for policy 1, policy_version 31940 (0.0007) [2023-10-12 21:18:44,573][44959] Updated weights for policy 1, policy_version 31950 (0.0008) [2023-10-12 21:18:44,950][44959] Updated weights for policy 1, policy_version 31960 (0.0009) [2023-10-12 21:18:46,305][44958] Updated weights for policy 0, policy_version 31780 (0.0008) [2023-10-12 21:18:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65273856. Throughput: 0: 1635.0, 1: 1647.3. Samples: 16331002. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 21:18:46,443][43579] Avg episode reward: [(0, '259.750'), (1, '278.180')] [2023-10-12 21:18:46,681][44958] Updated weights for policy 0, policy_version 31790 (0.0007) [2023-10-12 21:18:47,040][44958] Updated weights for policy 0, policy_version 31800 (0.0008) [2023-10-12 21:18:49,126][44959] Updated weights for policy 1, policy_version 31970 (0.0009) [2023-10-12 21:18:49,489][44959] Updated weights for policy 1, policy_version 31980 (0.0009) [2023-10-12 21:18:49,855][44959] Updated weights for policy 1, policy_version 31990 (0.0010) [2023-10-12 21:18:50,221][44959] Updated weights for policy 1, policy_version 32000 (0.0010) [2023-10-12 21:18:51,265][44958] Updated weights for policy 0, policy_version 31810 (0.0008) [2023-10-12 21:18:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65339392. Throughput: 0: 1630.2, 1: 1651.0. Samples: 16341024. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 21:18:51,444][43579] Avg episode reward: [(0, '256.220'), (1, '274.500')] [2023-10-12 21:18:51,651][44958] Updated weights for policy 0, policy_version 31820 (0.0010) [2023-10-12 21:18:52,017][44958] Updated weights for policy 0, policy_version 31830 (0.0009) [2023-10-12 21:18:52,398][44958] Updated weights for policy 0, policy_version 31840 (0.0009) [2023-10-12 21:18:54,307][44959] Updated weights for policy 1, policy_version 32010 (0.0007) [2023-10-12 21:18:54,673][44959] Updated weights for policy 1, policy_version 32020 (0.0008) [2023-10-12 21:18:55,044][44959] Updated weights for policy 1, policy_version 32030 (0.0007) [2023-10-12 21:18:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65404928. Throughput: 0: 1626.1, 1: 1640.8. Samples: 16360066. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 21:18:56,443][43579] Avg episode reward: [(0, '256.920'), (1, '275.920')] [2023-10-12 21:18:56,715][44958] Updated weights for policy 0, policy_version 31850 (0.0009) [2023-10-12 21:18:57,095][44958] Updated weights for policy 0, policy_version 31860 (0.0010) [2023-10-12 21:18:57,469][44958] Updated weights for policy 0, policy_version 31870 (0.0008) [2023-10-12 21:18:59,264][44959] Updated weights for policy 1, policy_version 32040 (0.0008) [2023-10-12 21:18:59,635][44959] Updated weights for policy 1, policy_version 32050 (0.0008) [2023-10-12 21:18:59,996][44959] Updated weights for policy 1, policy_version 32060 (0.0009) [2023-10-12 21:19:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65470464. Throughput: 0: 1631.0, 1: 1660.2. Samples: 16380304. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:19:01,443][43579] Avg episode reward: [(0, '265.770'), (1, '280.220')] [2023-10-12 21:19:01,566][44958] Updated weights for policy 0, policy_version 31880 (0.0007) [2023-10-12 21:19:01,938][44958] Updated weights for policy 0, policy_version 31890 (0.0008) [2023-10-12 21:19:02,312][44958] Updated weights for policy 0, policy_version 31900 (0.0008) [2023-10-12 21:19:04,100][44959] Updated weights for policy 1, policy_version 32070 (0.0008) [2023-10-12 21:19:04,458][44959] Updated weights for policy 1, policy_version 32080 (0.0008) [2023-10-12 21:19:04,831][44959] Updated weights for policy 1, policy_version 32090 (0.0009) [2023-10-12 21:19:06,339][44958] Updated weights for policy 0, policy_version 31910 (0.0010) [2023-10-12 21:19:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65536000. Throughput: 0: 1627.2, 1: 1666.7. Samples: 16390598. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:19:06,443][43579] Avg episode reward: [(0, '267.460'), (1, '279.450')] [2023-10-12 21:19:06,707][44958] Updated weights for policy 0, policy_version 31920 (0.0009) [2023-10-12 21:19:07,083][44958] Updated weights for policy 0, policy_version 31930 (0.0008) [2023-10-12 21:19:08,906][44959] Updated weights for policy 1, policy_version 32100 (0.0008) [2023-10-12 21:19:09,281][44959] Updated weights for policy 1, policy_version 32110 (0.0007) [2023-10-12 21:19:09,662][44959] Updated weights for policy 1, policy_version 32120 (0.0008) [2023-10-12 21:19:11,373][44958] Updated weights for policy 0, policy_version 31940 (0.0008) [2023-10-12 21:19:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65601536. Throughput: 0: 1634.8, 1: 1649.7. Samples: 16410062. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:19:11,443][43579] Avg episode reward: [(0, '268.190'), (1, '276.620')] [2023-10-12 21:19:11,768][44958] Updated weights for policy 0, policy_version 31950 (0.0010) [2023-10-12 21:19:12,139][44958] Updated weights for policy 0, policy_version 31960 (0.0008) [2023-10-12 21:19:13,799][44959] Updated weights for policy 1, policy_version 32130 (0.0008) [2023-10-12 21:19:14,171][44959] Updated weights for policy 1, policy_version 32140 (0.0007) [2023-10-12 21:19:14,536][44959] Updated weights for policy 1, policy_version 32150 (0.0007) [2023-10-12 21:19:14,912][44959] Updated weights for policy 1, policy_version 32160 (0.0010) [2023-10-12 21:19:16,193][44958] Updated weights for policy 0, policy_version 31970 (0.0009) [2023-10-12 21:19:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65667072. Throughput: 0: 1633.8, 1: 1665.2. Samples: 16430166. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:19:16,443][43579] Avg episode reward: [(0, '263.110'), (1, '275.680')] [2023-10-12 21:19:16,577][44958] Updated weights for policy 0, policy_version 31980 (0.0009) [2023-10-12 21:19:16,949][44958] Updated weights for policy 0, policy_version 31990 (0.0009) [2023-10-12 21:19:17,322][44958] Updated weights for policy 0, policy_version 32000 (0.0008) [2023-10-12 21:19:18,908][44959] Updated weights for policy 1, policy_version 32170 (0.0009) [2023-10-12 21:19:19,273][44959] Updated weights for policy 1, policy_version 32180 (0.0008) [2023-10-12 21:19:19,652][44959] Updated weights for policy 1, policy_version 32190 (0.0008) [2023-10-12 21:19:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65732608. Throughput: 0: 1634.4, 1: 1654.4. Samples: 16439828. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:19:21,443][43579] Avg episode reward: [(0, '268.530'), (1, '276.190')] [2023-10-12 21:19:21,497][44958] Updated weights for policy 0, policy_version 32010 (0.0008) [2023-10-12 21:19:21,859][44958] Updated weights for policy 0, policy_version 32020 (0.0008) [2023-10-12 21:19:22,238][44958] Updated weights for policy 0, policy_version 32030 (0.0009) [2023-10-12 21:19:23,824][44959] Updated weights for policy 1, policy_version 32200 (0.0009) [2023-10-12 21:19:24,192][44959] Updated weights for policy 1, policy_version 32210 (0.0010) [2023-10-12 21:19:24,572][44959] Updated weights for policy 1, policy_version 32220 (0.0010) [2023-10-12 21:19:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65798144. Throughput: 0: 1633.8, 1: 1655.4. Samples: 16459404. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-12 21:19:26,443][44958] Updated weights for policy 0, policy_version 32040 (0.0009) [2023-10-12 21:19:26,443][43579] Avg episode reward: [(0, '271.510'), (1, '274.450')] [2023-10-12 21:19:26,807][44958] Updated weights for policy 0, policy_version 32050 (0.0010) [2023-10-12 21:19:27,175][44958] Updated weights for policy 0, policy_version 32060 (0.0009) [2023-10-12 21:19:28,759][44959] Updated weights for policy 1, policy_version 32230 (0.0008) [2023-10-12 21:19:29,123][44959] Updated weights for policy 1, policy_version 32240 (0.0007) [2023-10-12 21:19:29,505][44959] Updated weights for policy 1, policy_version 32250 (0.0008) [2023-10-12 21:19:31,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 65863680. Throughput: 0: 1636.6, 1: 1671.1. Samples: 16479848. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-12 21:19:31,444][43579] Avg episode reward: [(0, '273.550'), (1, '274.240')] [2023-10-12 21:19:31,532][44958] Updated weights for policy 0, policy_version 32070 (0.0010) [2023-10-12 21:19:31,908][44958] Updated weights for policy 0, policy_version 32080 (0.0009) [2023-10-12 21:19:32,284][44958] Updated weights for policy 0, policy_version 32090 (0.0008) [2023-10-12 21:19:33,517][44959] Updated weights for policy 1, policy_version 32260 (0.0007) [2023-10-12 21:19:33,876][44959] Updated weights for policy 1, policy_version 32270 (0.0010) [2023-10-12 21:19:34,243][44959] Updated weights for policy 1, policy_version 32280 (0.0007) [2023-10-12 21:19:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 65929216. Throughput: 0: 1638.0, 1: 1652.0. Samples: 16489070. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-12 21:19:36,443][43579] Avg episode reward: [(0, '270.520'), (1, '277.400')] [2023-10-12 21:19:36,523][44958] Updated weights for policy 0, policy_version 32100 (0.0007) [2023-10-12 21:19:36,901][44958] Updated weights for policy 0, policy_version 32110 (0.0009) [2023-10-12 21:19:37,269][44958] Updated weights for policy 0, policy_version 32120 (0.0008) [2023-10-12 21:19:38,254][44959] Updated weights for policy 1, policy_version 32290 (0.0007) [2023-10-12 21:19:38,612][44959] Updated weights for policy 1, policy_version 32300 (0.0008) [2023-10-12 21:19:38,975][44959] Updated weights for policy 1, policy_version 32310 (0.0008) [2023-10-12 21:19:39,343][44959] Updated weights for policy 1, policy_version 32320 (0.0009) [2023-10-12 21:19:41,182][44958] Updated weights for policy 0, policy_version 32130 (0.0009) [2023-10-12 21:19:41,442][43579] Fps is (10 sec: 13107.8, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 65994752. Throughput: 0: 1642.4, 1: 1668.3. Samples: 16509048. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-12 21:19:41,443][43579] Avg episode reward: [(0, '264.140'), (1, '282.870')] [2023-10-12 21:19:41,551][44958] Updated weights for policy 0, policy_version 32140 (0.0008) [2023-10-12 21:19:41,919][44958] Updated weights for policy 0, policy_version 32150 (0.0008) [2023-10-12 21:19:42,292][44958] Updated weights for policy 0, policy_version 32160 (0.0007) [2023-10-12 21:19:43,623][44959] Updated weights for policy 1, policy_version 32330 (0.0010) [2023-10-12 21:19:44,004][44959] Updated weights for policy 1, policy_version 32340 (0.0009) [2023-10-12 21:19:44,378][44959] Updated weights for policy 1, policy_version 32350 (0.0009) [2023-10-12 21:19:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66060288. Throughput: 0: 1644.4, 1: 1666.2. Samples: 16529284. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-12 21:19:46,444][43579] Avg episode reward: [(0, '264.280'), (1, '282.380')] [2023-10-12 21:19:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000032352_33128448.pth... [2023-10-12 21:19:46,487][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000030816_31555584.pth [2023-10-12 21:19:46,582][44958] Updated weights for policy 0, policy_version 32170 (0.0011) [2023-10-12 21:19:46,958][44958] Updated weights for policy 0, policy_version 32180 (0.0008) [2023-10-12 21:19:47,331][44958] Updated weights for policy 0, policy_version 32190 (0.0010) [2023-10-12 21:19:47,409][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000032192_32964608.pth... [2023-10-12 21:19:47,439][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000030624_31358976.pth [2023-10-12 21:19:48,491][44959] Updated weights for policy 1, policy_version 32360 (0.0009) [2023-10-12 21:19:48,857][44959] Updated weights for policy 1, policy_version 32370 (0.0010) [2023-10-12 21:19:49,229][44959] Updated weights for policy 1, policy_version 32380 (0.0008) [2023-10-12 21:19:51,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66125824. Throughput: 0: 1643.2, 1: 1642.7. Samples: 16538466. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:19:51,444][43579] Avg episode reward: [(0, '262.660'), (1, '281.140')] [2023-10-12 21:19:51,622][44958] Updated weights for policy 0, policy_version 32200 (0.0007) [2023-10-12 21:19:51,991][44958] Updated weights for policy 0, policy_version 32210 (0.0009) [2023-10-12 21:19:52,353][44958] Updated weights for policy 0, policy_version 32220 (0.0009) [2023-10-12 21:19:53,536][44959] Updated weights for policy 1, policy_version 32390 (0.0010) [2023-10-12 21:19:53,903][44959] Updated weights for policy 1, policy_version 32400 (0.0010) [2023-10-12 21:19:54,283][44959] Updated weights for policy 1, policy_version 32410 (0.0008) [2023-10-12 21:19:56,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66191360. Throughput: 0: 1637.8, 1: 1656.1. Samples: 16558288. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:19:56,443][43579] Avg episode reward: [(0, '260.280'), (1, '279.870')] [2023-10-12 21:19:56,549][44958] Updated weights for policy 0, policy_version 32230 (0.0008) [2023-10-12 21:19:56,933][44958] Updated weights for policy 0, policy_version 32240 (0.0008) [2023-10-12 21:19:57,298][44958] Updated weights for policy 0, policy_version 32250 (0.0008) [2023-10-12 21:19:58,263][44959] Updated weights for policy 1, policy_version 32420 (0.0007) [2023-10-12 21:19:58,624][44959] Updated weights for policy 1, policy_version 32430 (0.0007) [2023-10-12 21:19:58,989][44959] Updated weights for policy 1, policy_version 32440 (0.0010) [2023-10-12 21:20:01,331][44958] Updated weights for policy 0, policy_version 32260 (0.0008) [2023-10-12 21:20:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66256896. Throughput: 0: 1641.9, 1: 1661.6. Samples: 16578822. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:20:01,443][43579] Avg episode reward: [(0, '259.950'), (1, '281.500')] [2023-10-12 21:20:01,708][44958] Updated weights for policy 0, policy_version 32270 (0.0009) [2023-10-12 21:20:02,083][44958] Updated weights for policy 0, policy_version 32280 (0.0008) [2023-10-12 21:20:03,165][44959] Updated weights for policy 1, policy_version 32450 (0.0009) [2023-10-12 21:20:03,536][44959] Updated weights for policy 1, policy_version 32460 (0.0010) [2023-10-12 21:20:03,893][44959] Updated weights for policy 1, policy_version 32470 (0.0011) [2023-10-12 21:20:04,260][44959] Updated weights for policy 1, policy_version 32480 (0.0010) [2023-10-12 21:20:06,118][44958] Updated weights for policy 0, policy_version 32290 (0.0008) [2023-10-12 21:20:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66322432. Throughput: 0: 1640.6, 1: 1650.7. Samples: 16587936. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:20:06,443][43579] Avg episode reward: [(0, '257.100'), (1, '280.450')] [2023-10-12 21:20:06,482][44958] Updated weights for policy 0, policy_version 32300 (0.0009) [2023-10-12 21:20:06,857][44958] Updated weights for policy 0, policy_version 32310 (0.0008) [2023-10-12 21:20:07,221][44958] Updated weights for policy 0, policy_version 32320 (0.0007) [2023-10-12 21:20:08,426][44959] Updated weights for policy 1, policy_version 32490 (0.0008) [2023-10-12 21:20:08,795][44959] Updated weights for policy 1, policy_version 32500 (0.0010) [2023-10-12 21:20:09,159][44959] Updated weights for policy 1, policy_version 32510 (0.0010) [2023-10-12 21:20:11,370][44958] Updated weights for policy 0, policy_version 32330 (0.0007) [2023-10-12 21:20:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66387968. Throughput: 0: 1648.4, 1: 1662.4. Samples: 16608390. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:20:11,443][43579] Avg episode reward: [(0, '264.260'), (1, '278.840')] [2023-10-12 21:20:11,751][44958] Updated weights for policy 0, policy_version 32340 (0.0008) [2023-10-12 21:20:12,132][44958] Updated weights for policy 0, policy_version 32350 (0.0009) [2023-10-12 21:20:13,191][44959] Updated weights for policy 1, policy_version 32520 (0.0008) [2023-10-12 21:20:13,560][44959] Updated weights for policy 1, policy_version 32530 (0.0009) [2023-10-12 21:20:13,935][44959] Updated weights for policy 1, policy_version 32540 (0.0010) [2023-10-12 21:20:16,216][44958] Updated weights for policy 0, policy_version 32360 (0.0008) [2023-10-12 21:20:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66453504. Throughput: 0: 1644.9, 1: 1660.0. Samples: 16628566. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:20:16,443][43579] Avg episode reward: [(0, '265.520'), (1, '275.400')] [2023-10-12 21:20:16,586][44958] Updated weights for policy 0, policy_version 32370 (0.0009) [2023-10-12 21:20:16,965][44958] Updated weights for policy 0, policy_version 32380 (0.0009) [2023-10-12 21:20:18,200][44959] Updated weights for policy 1, policy_version 32550 (0.0010) [2023-10-12 21:20:18,565][44959] Updated weights for policy 1, policy_version 32560 (0.0010) [2023-10-12 21:20:18,934][44959] Updated weights for policy 1, policy_version 32570 (0.0009) [2023-10-12 21:20:21,261][44958] Updated weights for policy 0, policy_version 32390 (0.0010) [2023-10-12 21:20:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66519040. Throughput: 0: 1650.8, 1: 1652.0. Samples: 16637696. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:20:21,443][43579] Avg episode reward: [(0, '265.500'), (1, '278.870')] [2023-10-12 21:20:21,624][44958] Updated weights for policy 0, policy_version 32400 (0.0009) [2023-10-12 21:20:21,989][44958] Updated weights for policy 0, policy_version 32410 (0.0009) [2023-10-12 21:20:23,139][44959] Updated weights for policy 1, policy_version 32580 (0.0008) [2023-10-12 21:20:23,508][44959] Updated weights for policy 1, policy_version 32590 (0.0010) [2023-10-12 21:20:23,875][44959] Updated weights for policy 1, policy_version 32600 (0.0008) [2023-10-12 21:20:26,194][44958] Updated weights for policy 0, policy_version 32420 (0.0010) [2023-10-12 21:20:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66584576. Throughput: 0: 1650.0, 1: 1656.0. Samples: 16657820. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:20:26,443][43579] Avg episode reward: [(0, '262.450'), (1, '280.010')] [2023-10-12 21:20:26,568][44958] Updated weights for policy 0, policy_version 32430 (0.0007) [2023-10-12 21:20:26,937][44958] Updated weights for policy 0, policy_version 32440 (0.0008) [2023-10-12 21:20:28,162][44959] Updated weights for policy 1, policy_version 32610 (0.0007) [2023-10-12 21:20:28,584][44959] Updated weights for policy 1, policy_version 32620 (0.0007) [2023-10-12 21:20:28,951][44959] Updated weights for policy 1, policy_version 32630 (0.0008) [2023-10-12 21:20:29,316][44959] Updated weights for policy 1, policy_version 32640 (0.0007) [2023-10-12 21:20:30,880][44958] Updated weights for policy 0, policy_version 32450 (0.0010) [2023-10-12 21:20:31,246][44958] Updated weights for policy 0, policy_version 32460 (0.0007) [2023-10-12 21:20:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 66650112. Throughput: 0: 1646.3, 1: 1647.0. Samples: 16677480. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:20:31,443][43579] Avg episode reward: [(0, '264.210'), (1, '271.800')] [2023-10-12 21:20:31,632][44958] Updated weights for policy 0, policy_version 32470 (0.0008) [2023-10-12 21:20:32,004][44958] Updated weights for policy 0, policy_version 32480 (0.0007) [2023-10-12 21:20:33,539][44959] Updated weights for policy 1, policy_version 32650 (0.0007) [2023-10-12 21:20:33,918][44959] Updated weights for policy 1, policy_version 32660 (0.0009) [2023-10-12 21:20:34,276][44959] Updated weights for policy 1, policy_version 32670 (0.0011) [2023-10-12 21:20:35,942][44958] Updated weights for policy 0, policy_version 32490 (0.0009) [2023-10-12 21:20:36,319][44958] Updated weights for policy 0, policy_version 32500 (0.0007) [2023-10-12 21:20:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66715648. Throughput: 0: 1656.0, 1: 1644.6. Samples: 16686994. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:20:36,444][43579] Avg episode reward: [(0, '269.080'), (1, '272.760')] [2023-10-12 21:20:36,705][44958] Updated weights for policy 0, policy_version 32510 (0.0008) [2023-10-12 21:20:38,330][44959] Updated weights for policy 1, policy_version 32680 (0.0008) [2023-10-12 21:20:38,697][44959] Updated weights for policy 1, policy_version 32690 (0.0008) [2023-10-12 21:20:39,067][44959] Updated weights for policy 1, policy_version 32700 (0.0008) [2023-10-12 21:20:40,747][44958] Updated weights for policy 0, policy_version 32520 (0.0008) [2023-10-12 21:20:41,128][44958] Updated weights for policy 0, policy_version 32530 (0.0007) [2023-10-12 21:20:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 66781184. Throughput: 0: 1659.8, 1: 1645.8. Samples: 16707040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:20:41,444][43579] Avg episode reward: [(0, '270.330'), (1, '272.790')] [2023-10-12 21:20:41,504][44958] Updated weights for policy 0, policy_version 32540 (0.0008) [2023-10-12 21:20:43,126][44959] Updated weights for policy 1, policy_version 32710 (0.0008) [2023-10-12 21:20:43,494][44959] Updated weights for policy 1, policy_version 32720 (0.0009) [2023-10-12 21:20:43,859][44959] Updated weights for policy 1, policy_version 32730 (0.0009) [2023-10-12 21:20:45,681][44958] Updated weights for policy 0, policy_version 32550 (0.0010) [2023-10-12 21:20:46,059][44958] Updated weights for policy 0, policy_version 32560 (0.0008) [2023-10-12 21:20:46,425][44958] Updated weights for policy 0, policy_version 32570 (0.0009) [2023-10-12 21:20:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66846720. Throughput: 0: 1645.1, 1: 1642.7. Samples: 16726770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:20:46,444][43579] Avg episode reward: [(0, '269.740'), (1, '274.980')] [2023-10-12 21:20:48,029][44959] Updated weights for policy 1, policy_version 32740 (0.0011) [2023-10-12 21:20:48,390][44959] Updated weights for policy 1, policy_version 32750 (0.0008) [2023-10-12 21:20:48,759][44959] Updated weights for policy 1, policy_version 32760 (0.0010) [2023-10-12 21:20:50,792][44958] Updated weights for policy 0, policy_version 32580 (0.0008) [2023-10-12 21:20:51,171][44958] Updated weights for policy 0, policy_version 32590 (0.0008) [2023-10-12 21:20:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66912256. Throughput: 0: 1657.9, 1: 1640.1. Samples: 16736350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:20:51,444][43579] Avg episode reward: [(0, '273.050'), (1, '275.440')] [2023-10-12 21:20:51,543][44958] Updated weights for policy 0, policy_version 32600 (0.0008) [2023-10-12 21:20:52,728][44959] Updated weights for policy 1, policy_version 32770 (0.0008) [2023-10-12 21:20:53,100][44959] Updated weights for policy 1, policy_version 32780 (0.0007) [2023-10-12 21:20:53,459][44959] Updated weights for policy 1, policy_version 32790 (0.0009) [2023-10-12 21:20:53,834][44959] Updated weights for policy 1, policy_version 32800 (0.0008) [2023-10-12 21:20:55,797][44958] Updated weights for policy 0, policy_version 32610 (0.0009) [2023-10-12 21:20:56,165][44958] Updated weights for policy 0, policy_version 32620 (0.0008) [2023-10-12 21:20:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 66977792. Throughput: 0: 1646.5, 1: 1650.5. Samples: 16756756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:20:56,443][43579] Avg episode reward: [(0, '280.550'), (1, '274.450')] [2023-10-12 21:20:56,538][44958] Updated weights for policy 0, policy_version 32630 (0.0008) [2023-10-12 21:20:56,903][44958] Updated weights for policy 0, policy_version 32640 (0.0008) [2023-10-12 21:20:57,969][44959] Updated weights for policy 1, policy_version 32810 (0.0009) [2023-10-12 21:20:58,337][44959] Updated weights for policy 1, policy_version 32820 (0.0007) [2023-10-12 21:20:58,714][44959] Updated weights for policy 1, policy_version 32830 (0.0009) [2023-10-12 21:21:01,029][44958] Updated weights for policy 0, policy_version 32650 (0.0009) [2023-10-12 21:21:01,388][44958] Updated weights for policy 0, policy_version 32660 (0.0009) [2023-10-12 21:21:01,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 67043328. Throughput: 0: 1640.5, 1: 1647.0. Samples: 16776502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:21:01,443][43579] Avg episode reward: [(0, '272.430'), (1, '277.740')] [2023-10-12 21:21:01,761][44958] Updated weights for policy 0, policy_version 32670 (0.0007) [2023-10-12 21:21:02,958][44959] Updated weights for policy 1, policy_version 32840 (0.0007) [2023-10-12 21:21:03,324][44959] Updated weights for policy 1, policy_version 32850 (0.0008) [2023-10-12 21:21:03,696][44959] Updated weights for policy 1, policy_version 32860 (0.0010) [2023-10-12 21:21:06,043][44958] Updated weights for policy 0, policy_version 32680 (0.0007) [2023-10-12 21:21:06,414][44958] Updated weights for policy 0, policy_version 32690 (0.0008) [2023-10-12 21:21:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67108864. Throughput: 0: 1648.9, 1: 1645.5. Samples: 16785944. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:21:06,443][43579] Avg episode reward: [(0, '270.680'), (1, '277.760')] [2023-10-12 21:21:06,794][44958] Updated weights for policy 0, policy_version 32700 (0.0009) [2023-10-12 21:21:07,836][44959] Updated weights for policy 1, policy_version 32870 (0.0010) [2023-10-12 21:21:08,211][44959] Updated weights for policy 1, policy_version 32880 (0.0007) [2023-10-12 21:21:08,580][44959] Updated weights for policy 1, policy_version 32890 (0.0010) [2023-10-12 21:21:10,690][44958] Updated weights for policy 0, policy_version 32710 (0.0008) [2023-10-12 21:21:11,064][44958] Updated weights for policy 0, policy_version 32720 (0.0007) [2023-10-12 21:21:11,443][44958] Updated weights for policy 0, policy_version 32730 (0.0009) [2023-10-12 21:21:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67174400. Throughput: 0: 1650.6, 1: 1649.6. Samples: 16806332. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:21:11,444][43579] Avg episode reward: [(0, '272.940'), (1, '277.660')] [2023-10-12 21:21:12,905][44959] Updated weights for policy 1, policy_version 32900 (0.0009) [2023-10-12 21:21:13,321][44959] Updated weights for policy 1, policy_version 32910 (0.0007) [2023-10-12 21:21:13,693][44959] Updated weights for policy 1, policy_version 32920 (0.0007) [2023-10-12 21:21:15,871][44958] Updated weights for policy 0, policy_version 32740 (0.0008) [2023-10-12 21:21:16,235][44958] Updated weights for policy 0, policy_version 32750 (0.0008) [2023-10-12 21:21:16,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67239936. Throughput: 0: 1638.9, 1: 1655.2. Samples: 16825714. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:21:16,443][43579] Avg episode reward: [(0, '267.490'), (1, '276.050')] [2023-10-12 21:21:16,617][44958] Updated weights for policy 0, policy_version 32760 (0.0007) [2023-10-12 21:21:17,932][44959] Updated weights for policy 1, policy_version 32930 (0.0008) [2023-10-12 21:21:18,289][44959] Updated weights for policy 1, policy_version 32940 (0.0010) [2023-10-12 21:21:18,657][44959] Updated weights for policy 1, policy_version 32950 (0.0009) [2023-10-12 21:21:19,022][44959] Updated weights for policy 1, policy_version 32960 (0.0008) [2023-10-12 21:21:20,824][44958] Updated weights for policy 0, policy_version 32770 (0.0008) [2023-10-12 21:21:21,204][44958] Updated weights for policy 0, policy_version 32780 (0.0007) [2023-10-12 21:21:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67305472. Throughput: 0: 1640.3, 1: 1649.2. Samples: 16835018. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:21:21,443][43579] Avg episode reward: [(0, '264.870'), (1, '275.080')] [2023-10-12 21:21:21,573][44958] Updated weights for policy 0, policy_version 32790 (0.0007) [2023-10-12 21:21:21,941][44958] Updated weights for policy 0, policy_version 32800 (0.0009) [2023-10-12 21:21:22,992][44959] Updated weights for policy 1, policy_version 32970 (0.0008) [2023-10-12 21:21:23,356][44959] Updated weights for policy 1, policy_version 32980 (0.0008) [2023-10-12 21:21:23,725][44959] Updated weights for policy 1, policy_version 32990 (0.0008) [2023-10-12 21:21:25,882][44958] Updated weights for policy 0, policy_version 32810 (0.0010) [2023-10-12 21:21:26,268][44958] Updated weights for policy 0, policy_version 32820 (0.0010) [2023-10-12 21:21:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67371008. Throughput: 0: 1642.0, 1: 1656.1. Samples: 16855450. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:21:26,443][43579] Avg episode reward: [(0, '264.680'), (1, '276.590')] [2023-10-12 21:21:26,641][44958] Updated weights for policy 0, policy_version 32830 (0.0009) [2023-10-12 21:21:27,837][44959] Updated weights for policy 1, policy_version 33000 (0.0009) [2023-10-12 21:21:28,201][44959] Updated weights for policy 1, policy_version 33010 (0.0009) [2023-10-12 21:21:28,566][44959] Updated weights for policy 1, policy_version 33020 (0.0008) [2023-10-12 21:21:31,004][44958] Updated weights for policy 0, policy_version 32840 (0.0008) [2023-10-12 21:21:31,367][44958] Updated weights for policy 0, policy_version 32850 (0.0011) [2023-10-12 21:21:31,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 67436544. Throughput: 0: 1640.9, 1: 1653.2. Samples: 16875006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:21:31,444][43579] Avg episode reward: [(0, '272.950'), (1, '274.990')] [2023-10-12 21:21:31,737][44958] Updated weights for policy 0, policy_version 32860 (0.0007) [2023-10-12 21:21:32,879][44959] Updated weights for policy 1, policy_version 33030 (0.0007) [2023-10-12 21:21:33,245][44959] Updated weights for policy 1, policy_version 33040 (0.0009) [2023-10-12 21:21:33,605][44959] Updated weights for policy 1, policy_version 33050 (0.0008) [2023-10-12 21:21:35,938][44958] Updated weights for policy 0, policy_version 32870 (0.0008) [2023-10-12 21:21:36,307][44958] Updated weights for policy 0, policy_version 32880 (0.0008) [2023-10-12 21:21:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67502080. Throughput: 0: 1637.9, 1: 1648.9. Samples: 16884258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:21:36,443][43579] Avg episode reward: [(0, '262.460'), (1, '276.670')] [2023-10-12 21:21:36,691][44958] Updated weights for policy 0, policy_version 32890 (0.0008) [2023-10-12 21:21:37,529][44959] Updated weights for policy 1, policy_version 33060 (0.0009) [2023-10-12 21:21:37,893][44959] Updated weights for policy 1, policy_version 33070 (0.0007) [2023-10-12 21:21:38,259][44959] Updated weights for policy 1, policy_version 33080 (0.0007) [2023-10-12 21:21:40,859][44958] Updated weights for policy 0, policy_version 32900 (0.0008) [2023-10-12 21:21:41,239][44958] Updated weights for policy 0, policy_version 32910 (0.0007) [2023-10-12 21:21:41,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67567616. Throughput: 0: 1640.5, 1: 1649.2. Samples: 16904792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:21:41,444][43579] Avg episode reward: [(0, '262.370'), (1, '277.780')] [2023-10-12 21:21:41,606][44958] Updated weights for policy 0, policy_version 32920 (0.0008) [2023-10-12 21:21:42,409][44959] Updated weights for policy 1, policy_version 33090 (0.0007) [2023-10-12 21:21:42,782][44959] Updated weights for policy 1, policy_version 33100 (0.0008) [2023-10-12 21:21:43,152][44959] Updated weights for policy 1, policy_version 33110 (0.0009) [2023-10-12 21:21:43,513][44959] Updated weights for policy 1, policy_version 33120 (0.0009) [2023-10-12 21:21:45,732][44958] Updated weights for policy 0, policy_version 32930 (0.0009) [2023-10-12 21:21:46,109][44958] Updated weights for policy 0, policy_version 32940 (0.0010) [2023-10-12 21:21:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67633152. Throughput: 0: 1637.9, 1: 1651.7. Samples: 16924534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:21:46,444][43579] Avg episode reward: [(0, '265.210'), (1, '275.460')] [2023-10-12 21:21:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000033120_33914880.pth... [2023-10-12 21:21:46,476][44958] Updated weights for policy 0, policy_version 32950 (0.0010) [2023-10-12 21:21:46,491][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000031584_32342016.pth [2023-10-12 21:21:46,844][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000032960_33751040.pth... [2023-10-12 21:21:46,845][44958] Updated weights for policy 0, policy_version 32960 (0.0011) [2023-10-12 21:21:46,881][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000031424_32178176.pth [2023-10-12 21:21:47,618][44959] Updated weights for policy 1, policy_version 33130 (0.0009) [2023-10-12 21:21:47,992][44959] Updated weights for policy 1, policy_version 33140 (0.0009) [2023-10-12 21:21:48,352][44959] Updated weights for policy 1, policy_version 33150 (0.0009) [2023-10-12 21:21:51,244][44958] Updated weights for policy 0, policy_version 32970 (0.0009) [2023-10-12 21:21:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67698688. Throughput: 0: 1631.0, 1: 1649.4. Samples: 16933564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:21:51,443][43579] Avg episode reward: [(0, '263.950'), (1, '277.240')] [2023-10-12 21:21:51,619][44958] Updated weights for policy 0, policy_version 32980 (0.0011) [2023-10-12 21:21:51,995][44958] Updated weights for policy 0, policy_version 32990 (0.0007) [2023-10-12 21:21:52,665][44959] Updated weights for policy 1, policy_version 33160 (0.0008) [2023-10-12 21:21:53,030][44959] Updated weights for policy 1, policy_version 33170 (0.0007) [2023-10-12 21:21:53,400][44959] Updated weights for policy 1, policy_version 33180 (0.0009) [2023-10-12 21:21:56,167][44958] Updated weights for policy 0, policy_version 33000 (0.0010) [2023-10-12 21:21:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67764224. Throughput: 0: 1631.3, 1: 1643.2. Samples: 16953682. Policy #0 lag: (min: 14.0, avg: 15.9, max: 46.0) [2023-10-12 21:21:56,444][43579] Avg episode reward: [(0, '255.640'), (1, '280.780')] [2023-10-12 21:21:56,545][44958] Updated weights for policy 0, policy_version 33010 (0.0008) [2023-10-12 21:21:56,917][44958] Updated weights for policy 0, policy_version 33020 (0.0009) [2023-10-12 21:21:57,740][44959] Updated weights for policy 1, policy_version 33190 (0.0009) [2023-10-12 21:21:58,117][44959] Updated weights for policy 1, policy_version 33200 (0.0008) [2023-10-12 21:21:58,479][44959] Updated weights for policy 1, policy_version 33210 (0.0010) [2023-10-12 21:22:01,151][44958] Updated weights for policy 0, policy_version 33030 (0.0008) [2023-10-12 21:22:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67829760. Throughput: 0: 1640.9, 1: 1649.4. Samples: 16973778. Policy #0 lag: (min: 14.0, avg: 15.9, max: 46.0) [2023-10-12 21:22:01,443][43579] Avg episode reward: [(0, '262.800'), (1, '284.440')] [2023-10-12 21:22:01,522][44958] Updated weights for policy 0, policy_version 33040 (0.0007) [2023-10-12 21:22:01,896][44958] Updated weights for policy 0, policy_version 33050 (0.0010) [2023-10-12 21:22:02,674][44959] Updated weights for policy 1, policy_version 33220 (0.0009) [2023-10-12 21:22:03,068][44959] Updated weights for policy 1, policy_version 33230 (0.0008) [2023-10-12 21:22:03,449][44959] Updated weights for policy 1, policy_version 33240 (0.0009) [2023-10-12 21:22:05,952][44958] Updated weights for policy 0, policy_version 33060 (0.0009) [2023-10-12 21:22:06,320][44958] Updated weights for policy 0, policy_version 33070 (0.0008) [2023-10-12 21:22:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67895296. Throughput: 0: 1637.8, 1: 1649.1. Samples: 16982928. Policy #0 lag: (min: 14.0, avg: 15.9, max: 46.0) [2023-10-12 21:22:06,444][43579] Avg episode reward: [(0, '270.210'), (1, '278.440')] [2023-10-12 21:22:06,700][44958] Updated weights for policy 0, policy_version 33080 (0.0008) [2023-10-12 21:22:07,473][44959] Updated weights for policy 1, policy_version 33250 (0.0009) [2023-10-12 21:22:07,845][44959] Updated weights for policy 1, policy_version 33260 (0.0007) [2023-10-12 21:22:08,217][44959] Updated weights for policy 1, policy_version 33270 (0.0011) [2023-10-12 21:22:08,585][44959] Updated weights for policy 1, policy_version 33280 (0.0010) [2023-10-12 21:22:11,082][44958] Updated weights for policy 0, policy_version 33090 (0.0007) [2023-10-12 21:22:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 67960832. Throughput: 0: 1631.6, 1: 1646.0. Samples: 17002942. Policy #0 lag: (min: 14.0, avg: 15.9, max: 46.0) [2023-10-12 21:22:11,443][43579] Avg episode reward: [(0, '271.730'), (1, '278.070')] [2023-10-12 21:22:11,460][44958] Updated weights for policy 0, policy_version 33100 (0.0009) [2023-10-12 21:22:11,821][44958] Updated weights for policy 0, policy_version 33110 (0.0009) [2023-10-12 21:22:12,200][44958] Updated weights for policy 0, policy_version 33120 (0.0010) [2023-10-12 21:22:12,805][44959] Updated weights for policy 1, policy_version 33290 (0.0009) [2023-10-12 21:22:13,182][44959] Updated weights for policy 1, policy_version 33300 (0.0009) [2023-10-12 21:22:13,550][44959] Updated weights for policy 1, policy_version 33310 (0.0008) [2023-10-12 21:22:16,429][44958] Updated weights for policy 0, policy_version 33130 (0.0009) [2023-10-12 21:22:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 68026368. Throughput: 0: 1644.0, 1: 1649.2. Samples: 17023198. Policy #0 lag: (min: 14.0, avg: 15.9, max: 46.0) [2023-10-12 21:22:16,444][43579] Avg episode reward: [(0, '268.050'), (1, '278.750')] [2023-10-12 21:22:16,810][44958] Updated weights for policy 0, policy_version 33140 (0.0007) [2023-10-12 21:22:17,177][44958] Updated weights for policy 0, policy_version 33150 (0.0008) [2023-10-12 21:22:17,524][44959] Updated weights for policy 1, policy_version 33320 (0.0008) [2023-10-12 21:22:17,889][44959] Updated weights for policy 1, policy_version 33330 (0.0009) [2023-10-12 21:22:18,256][44959] Updated weights for policy 1, policy_version 33340 (0.0009) [2023-10-12 21:22:21,211][44958] Updated weights for policy 0, policy_version 33160 (0.0009) [2023-10-12 21:22:21,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 68091904. Throughput: 0: 1638.6, 1: 1650.3. Samples: 17032258. Policy #0 lag: (min: 9.0, avg: 22.7, max: 41.0) [2023-10-12 21:22:21,444][43579] Avg episode reward: [(0, '271.990'), (1, '276.000')] [2023-10-12 21:22:21,580][44958] Updated weights for policy 0, policy_version 33170 (0.0009) [2023-10-12 21:22:21,951][44958] Updated weights for policy 0, policy_version 33180 (0.0009) [2023-10-12 21:22:22,505][44959] Updated weights for policy 1, policy_version 33350 (0.0009) [2023-10-12 21:22:22,865][44959] Updated weights for policy 1, policy_version 33360 (0.0008) [2023-10-12 21:22:23,240][44959] Updated weights for policy 1, policy_version 33370 (0.0007) [2023-10-12 21:22:25,927][44958] Updated weights for policy 0, policy_version 33190 (0.0009) [2023-10-12 21:22:26,294][44958] Updated weights for policy 0, policy_version 33200 (0.0010) [2023-10-12 21:22:26,443][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68157440. Throughput: 0: 1641.5, 1: 1644.4. Samples: 17052656. Policy #0 lag: (min: 9.0, avg: 22.7, max: 41.0) [2023-10-12 21:22:26,444][43579] Avg episode reward: [(0, '272.870'), (1, '276.020')] [2023-10-12 21:22:26,674][44958] Updated weights for policy 0, policy_version 33210 (0.0007) [2023-10-12 21:22:27,235][44959] Updated weights for policy 1, policy_version 33380 (0.0009) [2023-10-12 21:22:27,607][44959] Updated weights for policy 1, policy_version 33390 (0.0009) [2023-10-12 21:22:27,979][44959] Updated weights for policy 1, policy_version 33400 (0.0009) [2023-10-12 21:22:30,756][44958] Updated weights for policy 0, policy_version 33220 (0.0009) [2023-10-12 21:22:31,137][44958] Updated weights for policy 0, policy_version 33230 (0.0007) [2023-10-12 21:22:31,443][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 68222976. Throughput: 0: 1643.2, 1: 1647.2. Samples: 17072600. Policy #0 lag: (min: 9.0, avg: 22.7, max: 41.0) [2023-10-12 21:22:31,443][43579] Avg episode reward: [(0, '271.400'), (1, '275.080')] [2023-10-12 21:22:31,516][44958] Updated weights for policy 0, policy_version 33240 (0.0007) [2023-10-12 21:22:32,205][44959] Updated weights for policy 1, policy_version 33410 (0.0009) [2023-10-12 21:22:32,573][44959] Updated weights for policy 1, policy_version 33420 (0.0007) [2023-10-12 21:22:32,946][44959] Updated weights for policy 1, policy_version 33430 (0.0011) [2023-10-12 21:22:33,308][44959] Updated weights for policy 1, policy_version 33440 (0.0008) [2023-10-12 21:22:35,544][44958] Updated weights for policy 0, policy_version 33250 (0.0007) [2023-10-12 21:22:35,912][44958] Updated weights for policy 0, policy_version 33260 (0.0011) [2023-10-12 21:22:36,289][44958] Updated weights for policy 0, policy_version 33270 (0.0010) [2023-10-12 21:22:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68288512. Throughput: 0: 1650.5, 1: 1647.6. Samples: 17081978. Policy #0 lag: (min: 9.0, avg: 22.7, max: 41.0) [2023-10-12 21:22:36,443][43579] Avg episode reward: [(0, '270.510'), (1, '276.040')] [2023-10-12 21:22:36,654][44958] Updated weights for policy 0, policy_version 33280 (0.0009) [2023-10-12 21:22:37,446][44959] Updated weights for policy 1, policy_version 33450 (0.0009) [2023-10-12 21:22:37,805][44959] Updated weights for policy 1, policy_version 33460 (0.0009) [2023-10-12 21:22:38,175][44959] Updated weights for policy 1, policy_version 33470 (0.0008) [2023-10-12 21:22:40,896][44958] Updated weights for policy 0, policy_version 33290 (0.0008) [2023-10-12 21:22:41,265][44958] Updated weights for policy 0, policy_version 33300 (0.0008) [2023-10-12 21:22:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68354048. Throughput: 0: 1646.8, 1: 1654.8. Samples: 17102258. Policy #0 lag: (min: 9.0, avg: 22.7, max: 41.0) [2023-10-12 21:22:41,444][43579] Avg episode reward: [(0, '270.060'), (1, '273.990')] [2023-10-12 21:22:41,633][44958] Updated weights for policy 0, policy_version 33310 (0.0008) [2023-10-12 21:22:42,314][44959] Updated weights for policy 1, policy_version 33480 (0.0008) [2023-10-12 21:22:42,689][44959] Updated weights for policy 1, policy_version 33490 (0.0010) [2023-10-12 21:22:43,060][44959] Updated weights for policy 1, policy_version 33500 (0.0007) [2023-10-12 21:22:45,801][44958] Updated weights for policy 0, policy_version 33320 (0.0009) [2023-10-12 21:22:46,182][44958] Updated weights for policy 0, policy_version 33330 (0.0008) [2023-10-12 21:22:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68419584. Throughput: 0: 1639.6, 1: 1656.4. Samples: 17122100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:22:46,444][43579] Avg episode reward: [(0, '270.950'), (1, '278.400')] [2023-10-12 21:22:46,549][44958] Updated weights for policy 0, policy_version 33340 (0.0010) [2023-10-12 21:22:47,121][44959] Updated weights for policy 1, policy_version 33510 (0.0007) [2023-10-12 21:22:47,486][44959] Updated weights for policy 1, policy_version 33520 (0.0007) [2023-10-12 21:22:47,860][44959] Updated weights for policy 1, policy_version 33530 (0.0009) [2023-10-12 21:22:50,831][44958] Updated weights for policy 0, policy_version 33350 (0.0008) [2023-10-12 21:22:51,214][44958] Updated weights for policy 0, policy_version 33360 (0.0007) [2023-10-12 21:22:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68485120. Throughput: 0: 1645.9, 1: 1657.6. Samples: 17131586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:22:51,443][43579] Avg episode reward: [(0, '272.750'), (1, '279.740')] [2023-10-12 21:22:51,587][44958] Updated weights for policy 0, policy_version 33370 (0.0007) [2023-10-12 21:22:52,168][44959] Updated weights for policy 1, policy_version 33540 (0.0009) [2023-10-12 21:22:52,562][44959] Updated weights for policy 1, policy_version 33550 (0.0010) [2023-10-12 21:22:52,936][44959] Updated weights for policy 1, policy_version 33560 (0.0009) [2023-10-12 21:22:55,859][44958] Updated weights for policy 0, policy_version 33380 (0.0007) [2023-10-12 21:22:56,236][44958] Updated weights for policy 0, policy_version 33390 (0.0008) [2023-10-12 21:22:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68550656. Throughput: 0: 1641.6, 1: 1660.4. Samples: 17151532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:22:56,443][43579] Avg episode reward: [(0, '276.650'), (1, '281.500')] [2023-10-12 21:22:56,609][44958] Updated weights for policy 0, policy_version 33400 (0.0007) [2023-10-12 21:22:57,171][44959] Updated weights for policy 1, policy_version 33570 (0.0009) [2023-10-12 21:22:57,535][44959] Updated weights for policy 1, policy_version 33580 (0.0010) [2023-10-12 21:22:57,897][44959] Updated weights for policy 1, policy_version 33590 (0.0010) [2023-10-12 21:22:58,256][44959] Updated weights for policy 1, policy_version 33600 (0.0007) [2023-10-12 21:23:00,716][44958] Updated weights for policy 0, policy_version 33410 (0.0009) [2023-10-12 21:23:01,099][44958] Updated weights for policy 0, policy_version 33420 (0.0010) [2023-10-12 21:23:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68616192. Throughput: 0: 1635.3, 1: 1650.0. Samples: 17171032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:23:01,443][43579] Avg episode reward: [(0, '276.750'), (1, '282.660')] [2023-10-12 21:23:01,471][44958] Updated weights for policy 0, policy_version 33430 (0.0010) [2023-10-12 21:23:01,841][44958] Updated weights for policy 0, policy_version 33440 (0.0008) [2023-10-12 21:23:02,419][44959] Updated weights for policy 1, policy_version 33610 (0.0011) [2023-10-12 21:23:02,787][44959] Updated weights for policy 1, policy_version 33620 (0.0011) [2023-10-12 21:23:03,165][44959] Updated weights for policy 1, policy_version 33630 (0.0010) [2023-10-12 21:23:05,975][44958] Updated weights for policy 0, policy_version 33450 (0.0009) [2023-10-12 21:23:06,345][44958] Updated weights for policy 0, policy_version 33460 (0.0009) [2023-10-12 21:23:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68681728. Throughput: 0: 1641.2, 1: 1650.1. Samples: 17180364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:23:06,444][43579] Avg episode reward: [(0, '276.540'), (1, '283.390')] [2023-10-12 21:23:06,720][44958] Updated weights for policy 0, policy_version 33470 (0.0008) [2023-10-12 21:23:07,226][44959] Updated weights for policy 1, policy_version 33640 (0.0009) [2023-10-12 21:23:07,594][44959] Updated weights for policy 1, policy_version 33650 (0.0009) [2023-10-12 21:23:07,962][44959] Updated weights for policy 1, policy_version 33660 (0.0009) [2023-10-12 21:23:10,913][44958] Updated weights for policy 0, policy_version 33480 (0.0010) [2023-10-12 21:23:11,276][44958] Updated weights for policy 0, policy_version 33490 (0.0010) [2023-10-12 21:23:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68747264. Throughput: 0: 1637.6, 1: 1656.0. Samples: 17200870. Policy #0 lag: (min: 14.0, avg: 16.6, max: 46.0) [2023-10-12 21:23:11,443][43579] Avg episode reward: [(0, '278.220'), (1, '283.320')] [2023-10-12 21:23:11,648][44958] Updated weights for policy 0, policy_version 33500 (0.0008) [2023-10-12 21:23:11,993][44959] Updated weights for policy 1, policy_version 33670 (0.0009) [2023-10-12 21:23:12,368][44959] Updated weights for policy 1, policy_version 33680 (0.0010) [2023-10-12 21:23:12,745][44959] Updated weights for policy 1, policy_version 33690 (0.0008) [2023-10-12 21:23:15,820][44958] Updated weights for policy 0, policy_version 33510 (0.0009) [2023-10-12 21:23:16,193][44958] Updated weights for policy 0, policy_version 33520 (0.0010) [2023-10-12 21:23:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68812800. Throughput: 0: 1635.3, 1: 1654.2. Samples: 17220626. Policy #0 lag: (min: 14.0, avg: 16.6, max: 46.0) [2023-10-12 21:23:16,444][43579] Avg episode reward: [(0, '280.760'), (1, '281.600')] [2023-10-12 21:23:16,562][44958] Updated weights for policy 0, policy_version 33530 (0.0011) [2023-10-12 21:23:16,987][44959] Updated weights for policy 1, policy_version 33700 (0.0008) [2023-10-12 21:23:17,364][44959] Updated weights for policy 1, policy_version 33710 (0.0007) [2023-10-12 21:23:17,738][44959] Updated weights for policy 1, policy_version 33720 (0.0008) [2023-10-12 21:23:20,608][44958] Updated weights for policy 0, policy_version 33540 (0.0008) [2023-10-12 21:23:20,987][44958] Updated weights for policy 0, policy_version 33550 (0.0008) [2023-10-12 21:23:21,357][44958] Updated weights for policy 0, policy_version 33560 (0.0007) [2023-10-12 21:23:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68878336. Throughput: 0: 1635.6, 1: 1655.2. Samples: 17230068. Policy #0 lag: (min: 14.0, avg: 16.6, max: 46.0) [2023-10-12 21:23:21,444][43579] Avg episode reward: [(0, '282.110'), (1, '275.370')] [2023-10-12 21:23:21,838][44959] Updated weights for policy 1, policy_version 33730 (0.0008) [2023-10-12 21:23:22,199][44959] Updated weights for policy 1, policy_version 33740 (0.0009) [2023-10-12 21:23:22,561][44959] Updated weights for policy 1, policy_version 33750 (0.0010) [2023-10-12 21:23:22,943][44959] Updated weights for policy 1, policy_version 33760 (0.0010) [2023-10-12 21:23:25,638][44958] Updated weights for policy 0, policy_version 33570 (0.0008) [2023-10-12 21:23:26,016][44958] Updated weights for policy 0, policy_version 33580 (0.0008) [2023-10-12 21:23:26,378][44958] Updated weights for policy 0, policy_version 33590 (0.0010) [2023-10-12 21:23:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 68943872. Throughput: 0: 1634.0, 1: 1652.3. Samples: 17250140. Policy #0 lag: (min: 14.0, avg: 16.6, max: 46.0) [2023-10-12 21:23:26,443][43579] Avg episode reward: [(0, '281.710'), (1, '272.490')] [2023-10-12 21:23:26,748][44958] Updated weights for policy 0, policy_version 33600 (0.0010) [2023-10-12 21:23:26,967][44959] Updated weights for policy 1, policy_version 33770 (0.0009) [2023-10-12 21:23:27,349][44959] Updated weights for policy 1, policy_version 33780 (0.0009) [2023-10-12 21:23:27,724][44959] Updated weights for policy 1, policy_version 33790 (0.0010) [2023-10-12 21:23:30,882][44958] Updated weights for policy 0, policy_version 33610 (0.0008) [2023-10-12 21:23:31,259][44958] Updated weights for policy 0, policy_version 33620 (0.0008) [2023-10-12 21:23:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69009408. Throughput: 0: 1634.7, 1: 1650.3. Samples: 17269924. Policy #0 lag: (min: 14.0, avg: 16.6, max: 46.0) [2023-10-12 21:23:31,443][43579] Avg episode reward: [(0, '284.120'), (1, '273.370')] [2023-10-12 21:23:31,629][44958] Updated weights for policy 0, policy_version 33630 (0.0008) [2023-10-12 21:23:31,943][44959] Updated weights for policy 1, policy_version 33800 (0.0008) [2023-10-12 21:23:32,313][44959] Updated weights for policy 1, policy_version 33810 (0.0008) [2023-10-12 21:23:32,689][44959] Updated weights for policy 1, policy_version 33820 (0.0010) [2023-10-12 21:23:35,878][44958] Updated weights for policy 0, policy_version 33640 (0.0007) [2023-10-12 21:23:36,252][44958] Updated weights for policy 0, policy_version 33650 (0.0009) [2023-10-12 21:23:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69074944. Throughput: 0: 1637.4, 1: 1647.7. Samples: 17279416. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-12 21:23:36,443][43579] Avg episode reward: [(0, '282.700'), (1, '277.240')] [2023-10-12 21:23:36,621][44958] Updated weights for policy 0, policy_version 33660 (0.0008) [2023-10-12 21:23:36,981][44959] Updated weights for policy 1, policy_version 33830 (0.0009) [2023-10-12 21:23:37,366][44959] Updated weights for policy 1, policy_version 33840 (0.0007) [2023-10-12 21:23:37,741][44959] Updated weights for policy 1, policy_version 33850 (0.0007) [2023-10-12 21:23:40,931][44958] Updated weights for policy 0, policy_version 33670 (0.0007) [2023-10-12 21:23:41,292][44958] Updated weights for policy 0, policy_version 33680 (0.0008) [2023-10-12 21:23:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69140480. Throughput: 0: 1639.4, 1: 1644.6. Samples: 17299312. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-12 21:23:41,443][43579] Avg episode reward: [(0, '282.980'), (1, '274.940')] [2023-10-12 21:23:41,674][44958] Updated weights for policy 0, policy_version 33690 (0.0009) [2023-10-12 21:23:41,993][44959] Updated weights for policy 1, policy_version 33860 (0.0008) [2023-10-12 21:23:42,367][44959] Updated weights for policy 1, policy_version 33870 (0.0008) [2023-10-12 21:23:42,729][44959] Updated weights for policy 1, policy_version 33880 (0.0007) [2023-10-12 21:23:45,601][44958] Updated weights for policy 0, policy_version 33700 (0.0007) [2023-10-12 21:23:46,003][44958] Updated weights for policy 0, policy_version 33710 (0.0010) [2023-10-12 21:23:46,373][44958] Updated weights for policy 0, policy_version 33720 (0.0010) [2023-10-12 21:23:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69206016. Throughput: 0: 1641.3, 1: 1657.1. Samples: 17319462. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-12 21:23:46,444][43579] Avg episode reward: [(0, '283.580'), (1, '271.660')] [2023-10-12 21:23:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000033888_34701312.pth... [2023-10-12 21:23:46,486][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000032352_33128448.pth [2023-10-12 21:23:46,666][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000033728_34537472.pth... [2023-10-12 21:23:46,704][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000032192_32964608.pth [2023-10-12 21:23:46,869][44959] Updated weights for policy 1, policy_version 33890 (0.0007) [2023-10-12 21:23:47,242][44959] Updated weights for policy 1, policy_version 33900 (0.0009) [2023-10-12 21:23:47,605][44959] Updated weights for policy 1, policy_version 33910 (0.0010) [2023-10-12 21:23:47,971][44959] Updated weights for policy 1, policy_version 33920 (0.0009) [2023-10-12 21:23:50,667][44958] Updated weights for policy 0, policy_version 33730 (0.0009) [2023-10-12 21:23:51,047][44958] Updated weights for policy 0, policy_version 33740 (0.0010) [2023-10-12 21:23:51,427][44958] Updated weights for policy 0, policy_version 33750 (0.0009) [2023-10-12 21:23:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69271552. Throughput: 0: 1643.6, 1: 1655.9. Samples: 17328840. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-12 21:23:51,444][43579] Avg episode reward: [(0, '272.180'), (1, '267.270')] [2023-10-12 21:23:51,799][44958] Updated weights for policy 0, policy_version 33760 (0.0009) [2023-10-12 21:23:52,265][44959] Updated weights for policy 1, policy_version 33930 (0.0008) [2023-10-12 21:23:52,625][44959] Updated weights for policy 1, policy_version 33940 (0.0011) [2023-10-12 21:23:53,002][44959] Updated weights for policy 1, policy_version 33950 (0.0009) [2023-10-12 21:23:55,909][44958] Updated weights for policy 0, policy_version 33770 (0.0007) [2023-10-12 21:23:56,277][44958] Updated weights for policy 0, policy_version 33780 (0.0008) [2023-10-12 21:23:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69337088. Throughput: 0: 1645.0, 1: 1652.9. Samples: 17349276. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-12 21:23:56,444][43579] Avg episode reward: [(0, '271.970'), (1, '267.800')] [2023-10-12 21:23:56,649][44958] Updated weights for policy 0, policy_version 33790 (0.0008) [2023-10-12 21:23:57,026][44959] Updated weights for policy 1, policy_version 33960 (0.0010) [2023-10-12 21:23:57,407][44959] Updated weights for policy 1, policy_version 33970 (0.0009) [2023-10-12 21:23:57,779][44959] Updated weights for policy 1, policy_version 33980 (0.0009) [2023-10-12 21:24:00,713][44958] Updated weights for policy 0, policy_version 33800 (0.0008) [2023-10-12 21:24:01,079][44958] Updated weights for policy 0, policy_version 33810 (0.0009) [2023-10-12 21:24:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69402624. Throughput: 0: 1643.3, 1: 1648.5. Samples: 17368754. Policy #0 lag: (min: 21.0, avg: 23.7, max: 53.0) [2023-10-12 21:24:01,443][43579] Avg episode reward: [(0, '268.950'), (1, '267.180')] [2023-10-12 21:24:01,454][44958] Updated weights for policy 0, policy_version 33820 (0.0008) [2023-10-12 21:24:02,033][44959] Updated weights for policy 1, policy_version 33990 (0.0008) [2023-10-12 21:24:02,401][44959] Updated weights for policy 1, policy_version 34000 (0.0010) [2023-10-12 21:24:02,771][44959] Updated weights for policy 1, policy_version 34010 (0.0009) [2023-10-12 21:24:05,568][44958] Updated weights for policy 0, policy_version 33830 (0.0009) [2023-10-12 21:24:05,935][44958] Updated weights for policy 0, policy_version 33840 (0.0009) [2023-10-12 21:24:06,304][44958] Updated weights for policy 0, policy_version 33850 (0.0011) [2023-10-12 21:24:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69468160. Throughput: 0: 1647.7, 1: 1648.1. Samples: 17378380. Policy #0 lag: (min: 21.0, avg: 23.7, max: 53.0) [2023-10-12 21:24:06,444][43579] Avg episode reward: [(0, '265.960'), (1, '269.090')] [2023-10-12 21:24:06,701][44959] Updated weights for policy 1, policy_version 34020 (0.0008) [2023-10-12 21:24:07,064][44959] Updated weights for policy 1, policy_version 34030 (0.0009) [2023-10-12 21:24:07,432][44959] Updated weights for policy 1, policy_version 34040 (0.0010) [2023-10-12 21:24:10,549][44958] Updated weights for policy 0, policy_version 33860 (0.0009) [2023-10-12 21:24:10,918][44958] Updated weights for policy 0, policy_version 33870 (0.0008) [2023-10-12 21:24:11,283][44958] Updated weights for policy 0, policy_version 33880 (0.0008) [2023-10-12 21:24:11,370][44959] Updated weights for policy 1, policy_version 34050 (0.0007) [2023-10-12 21:24:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69533696. Throughput: 0: 1650.0, 1: 1651.9. Samples: 17398728. Policy #0 lag: (min: 21.0, avg: 23.7, max: 53.0) [2023-10-12 21:24:11,444][43579] Avg episode reward: [(0, '264.280'), (1, '272.270')] [2023-10-12 21:24:11,730][44959] Updated weights for policy 1, policy_version 34060 (0.0009) [2023-10-12 21:24:12,100][44959] Updated weights for policy 1, policy_version 34070 (0.0009) [2023-10-12 21:24:12,465][44959] Updated weights for policy 1, policy_version 34080 (0.0009) [2023-10-12 21:24:15,524][44958] Updated weights for policy 0, policy_version 33890 (0.0008) [2023-10-12 21:24:15,897][44958] Updated weights for policy 0, policy_version 33900 (0.0007) [2023-10-12 21:24:16,275][44958] Updated weights for policy 0, policy_version 33910 (0.0009) [2023-10-12 21:24:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69599232. Throughput: 0: 1640.1, 1: 1650.6. Samples: 17418006. Policy #0 lag: (min: 21.0, avg: 23.7, max: 53.0) [2023-10-12 21:24:16,443][43579] Avg episode reward: [(0, '263.720'), (1, '275.360')] [2023-10-12 21:24:16,645][44958] Updated weights for policy 0, policy_version 33920 (0.0009) [2023-10-12 21:24:16,722][44959] Updated weights for policy 1, policy_version 34090 (0.0008) [2023-10-12 21:24:17,092][44959] Updated weights for policy 1, policy_version 34100 (0.0008) [2023-10-12 21:24:17,457][44959] Updated weights for policy 1, policy_version 34110 (0.0007) [2023-10-12 21:24:20,668][44958] Updated weights for policy 0, policy_version 33930 (0.0009) [2023-10-12 21:24:21,042][44958] Updated weights for policy 0, policy_version 33940 (0.0008) [2023-10-12 21:24:21,403][44958] Updated weights for policy 0, policy_version 33950 (0.0009) [2023-10-12 21:24:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69664768. Throughput: 0: 1638.7, 1: 1655.6. Samples: 17427660. Policy #0 lag: (min: 21.0, avg: 23.7, max: 53.0) [2023-10-12 21:24:21,443][43579] Avg episode reward: [(0, '268.380'), (1, '269.580')] [2023-10-12 21:24:21,558][44959] Updated weights for policy 1, policy_version 34120 (0.0009) [2023-10-12 21:24:21,923][44959] Updated weights for policy 1, policy_version 34130 (0.0010) [2023-10-12 21:24:22,288][44959] Updated weights for policy 1, policy_version 34140 (0.0011) [2023-10-12 21:24:25,671][44958] Updated weights for policy 0, policy_version 33960 (0.0008) [2023-10-12 21:24:26,046][44958] Updated weights for policy 0, policy_version 33970 (0.0008) [2023-10-12 21:24:26,415][44958] Updated weights for policy 0, policy_version 33980 (0.0008) [2023-10-12 21:24:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69730304. Throughput: 0: 1644.5, 1: 1657.0. Samples: 17447878. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) [2023-10-12 21:24:26,443][43579] Avg episode reward: [(0, '266.230'), (1, '273.540')] [2023-10-12 21:24:26,498][44959] Updated weights for policy 1, policy_version 34150 (0.0008) [2023-10-12 21:24:26,873][44959] Updated weights for policy 1, policy_version 34160 (0.0008) [2023-10-12 21:24:27,241][44959] Updated weights for policy 1, policy_version 34170 (0.0008) [2023-10-12 21:24:30,836][44958] Updated weights for policy 0, policy_version 33990 (0.0007) [2023-10-12 21:24:31,226][44958] Updated weights for policy 0, policy_version 34000 (0.0008) [2023-10-12 21:24:31,244][44959] Updated weights for policy 1, policy_version 34180 (0.0008) [2023-10-12 21:24:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69795840. Throughput: 0: 1630.4, 1: 1647.8. Samples: 17466980. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) [2023-10-12 21:24:31,444][43579] Avg episode reward: [(0, '269.930'), (1, '271.730')] [2023-10-12 21:24:31,585][44958] Updated weights for policy 0, policy_version 34010 (0.0007) [2023-10-12 21:24:31,610][44959] Updated weights for policy 1, policy_version 34190 (0.0007) [2023-10-12 21:24:31,972][44959] Updated weights for policy 1, policy_version 34200 (0.0010) [2023-10-12 21:24:35,773][44958] Updated weights for policy 0, policy_version 34020 (0.0009) [2023-10-12 21:24:36,083][44959] Updated weights for policy 1, policy_version 34210 (0.0010) [2023-10-12 21:24:36,146][44958] Updated weights for policy 0, policy_version 34030 (0.0011) [2023-10-12 21:24:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69861376. Throughput: 0: 1629.3, 1: 1652.3. Samples: 17476512. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) [2023-10-12 21:24:36,443][43579] Avg episode reward: [(0, '267.200'), (1, '266.580')] [2023-10-12 21:24:36,459][44959] Updated weights for policy 1, policy_version 34220 (0.0010) [2023-10-12 21:24:36,522][44958] Updated weights for policy 0, policy_version 34040 (0.0010) [2023-10-12 21:24:36,826][44959] Updated weights for policy 1, policy_version 34230 (0.0011) [2023-10-12 21:24:37,189][44959] Updated weights for policy 1, policy_version 34240 (0.0010) [2023-10-12 21:24:40,728][44958] Updated weights for policy 0, policy_version 34050 (0.0008) [2023-10-12 21:24:41,100][44958] Updated weights for policy 0, policy_version 34060 (0.0008) [2023-10-12 21:24:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69926912. Throughput: 0: 1626.4, 1: 1648.7. Samples: 17496656. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) [2023-10-12 21:24:41,443][43579] Avg episode reward: [(0, '274.170'), (1, '266.400')] [2023-10-12 21:24:41,476][44958] Updated weights for policy 0, policy_version 34070 (0.0007) [2023-10-12 21:24:41,592][44959] Updated weights for policy 1, policy_version 34250 (0.0009) [2023-10-12 21:24:41,840][44958] Updated weights for policy 0, policy_version 34080 (0.0008) [2023-10-12 21:24:41,963][44959] Updated weights for policy 1, policy_version 34260 (0.0009) [2023-10-12 21:24:42,336][44959] Updated weights for policy 1, policy_version 34270 (0.0010) [2023-10-12 21:24:46,120][44958] Updated weights for policy 0, policy_version 34090 (0.0007) [2023-10-12 21:24:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 69992448. Throughput: 0: 1628.2, 1: 1650.6. Samples: 17516300. Policy #0 lag: (min: 0.0, avg: 26.3, max: 32.0) [2023-10-12 21:24:46,443][43579] Avg episode reward: [(0, '274.050'), (1, '268.090')] [2023-10-12 21:24:46,487][44959] Updated weights for policy 1, policy_version 34280 (0.0009) [2023-10-12 21:24:46,497][44958] Updated weights for policy 0, policy_version 34100 (0.0008) [2023-10-12 21:24:46,853][44959] Updated weights for policy 1, policy_version 34290 (0.0009) [2023-10-12 21:24:46,868][44958] Updated weights for policy 0, policy_version 34110 (0.0008) [2023-10-12 21:24:47,220][44959] Updated weights for policy 1, policy_version 34300 (0.0009) [2023-10-12 21:24:51,145][44958] Updated weights for policy 0, policy_version 34120 (0.0008) [2023-10-12 21:24:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70057984. Throughput: 0: 1621.2, 1: 1650.4. Samples: 17525600. Policy #0 lag: (min: 10.0, avg: 12.9, max: 42.0) [2023-10-12 21:24:51,443][43579] Avg episode reward: [(0, '277.070'), (1, '267.750')] [2023-10-12 21:24:51,477][44959] Updated weights for policy 1, policy_version 34310 (0.0009) [2023-10-12 21:24:51,521][44958] Updated weights for policy 0, policy_version 34130 (0.0007) [2023-10-12 21:24:51,842][44959] Updated weights for policy 1, policy_version 34320 (0.0010) [2023-10-12 21:24:51,900][44958] Updated weights for policy 0, policy_version 34140 (0.0008) [2023-10-12 21:24:52,225][44959] Updated weights for policy 1, policy_version 34330 (0.0009) [2023-10-12 21:24:56,266][44958] Updated weights for policy 0, policy_version 34150 (0.0007) [2023-10-12 21:24:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70123520. Throughput: 0: 1619.9, 1: 1645.5. Samples: 17545670. Policy #0 lag: (min: 10.0, avg: 12.9, max: 42.0) [2023-10-12 21:24:56,444][43579] Avg episode reward: [(0, '276.200'), (1, '267.290')] [2023-10-12 21:24:56,528][44959] Updated weights for policy 1, policy_version 34340 (0.0010) [2023-10-12 21:24:56,630][44958] Updated weights for policy 0, policy_version 34160 (0.0007) [2023-10-12 21:24:56,895][44959] Updated weights for policy 1, policy_version 34350 (0.0009) [2023-10-12 21:24:57,000][44958] Updated weights for policy 0, policy_version 34170 (0.0008) [2023-10-12 21:24:57,267][44959] Updated weights for policy 1, policy_version 34360 (0.0008) [2023-10-12 21:25:00,968][44958] Updated weights for policy 0, policy_version 34180 (0.0009) [2023-10-12 21:25:01,336][44958] Updated weights for policy 0, policy_version 34190 (0.0011) [2023-10-12 21:25:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 70189056. Throughput: 0: 1635.5, 1: 1643.0. Samples: 17565536. Policy #0 lag: (min: 10.0, avg: 12.9, max: 42.0) [2023-10-12 21:25:01,444][43579] Avg episode reward: [(0, '275.170'), (1, '273.640')] [2023-10-12 21:25:01,469][44959] Updated weights for policy 1, policy_version 34370 (0.0009) [2023-10-12 21:25:01,712][44958] Updated weights for policy 0, policy_version 34200 (0.0009) [2023-10-12 21:25:01,837][44959] Updated weights for policy 1, policy_version 34380 (0.0009) [2023-10-12 21:25:02,201][44959] Updated weights for policy 1, policy_version 34390 (0.0009) [2023-10-12 21:25:02,567][44959] Updated weights for policy 1, policy_version 34400 (0.0010) [2023-10-12 21:25:05,675][44958] Updated weights for policy 0, policy_version 34210 (0.0008) [2023-10-12 21:25:06,038][44958] Updated weights for policy 0, policy_version 34220 (0.0010) [2023-10-12 21:25:06,416][44958] Updated weights for policy 0, policy_version 34230 (0.0010) [2023-10-12 21:25:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70254592. Throughput: 0: 1629.1, 1: 1640.0. Samples: 17574774. Policy #0 lag: (min: 10.0, avg: 12.9, max: 42.0) [2023-10-12 21:25:06,444][43579] Avg episode reward: [(0, '276.990'), (1, '266.290')] [2023-10-12 21:25:06,642][44959] Updated weights for policy 1, policy_version 34410 (0.0009) [2023-10-12 21:25:06,790][44958] Updated weights for policy 0, policy_version 34240 (0.0009) [2023-10-12 21:25:07,012][44959] Updated weights for policy 1, policy_version 34420 (0.0011) [2023-10-12 21:25:07,379][44959] Updated weights for policy 1, policy_version 34430 (0.0009) [2023-10-12 21:25:11,181][44958] Updated weights for policy 0, policy_version 34250 (0.0008) [2023-10-12 21:25:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70320128. Throughput: 0: 1621.4, 1: 1639.8. Samples: 17594634. Policy #0 lag: (min: 10.0, avg: 12.9, max: 42.0) [2023-10-12 21:25:11,443][43579] Avg episode reward: [(0, '274.020'), (1, '267.580')] [2023-10-12 21:25:11,548][44958] Updated weights for policy 0, policy_version 34260 (0.0011) [2023-10-12 21:25:11,867][44959] Updated weights for policy 1, policy_version 34440 (0.0009) [2023-10-12 21:25:11,923][44958] Updated weights for policy 0, policy_version 34270 (0.0009) [2023-10-12 21:25:12,244][44959] Updated weights for policy 1, policy_version 34450 (0.0011) [2023-10-12 21:25:12,615][44959] Updated weights for policy 1, policy_version 34460 (0.0008) [2023-10-12 21:25:16,186][44958] Updated weights for policy 0, policy_version 34280 (0.0009) [2023-10-12 21:25:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70385664. Throughput: 0: 1638.8, 1: 1640.5. Samples: 17614548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:25:16,443][43579] Avg episode reward: [(0, '273.170'), (1, '263.810')] [2023-10-12 21:25:16,562][44958] Updated weights for policy 0, policy_version 34290 (0.0009) [2023-10-12 21:25:16,595][44959] Updated weights for policy 1, policy_version 34470 (0.0009) [2023-10-12 21:25:16,941][44958] Updated weights for policy 0, policy_version 34300 (0.0009) [2023-10-12 21:25:16,971][44959] Updated weights for policy 1, policy_version 34480 (0.0008) [2023-10-12 21:25:17,330][44959] Updated weights for policy 1, policy_version 34490 (0.0007) [2023-10-12 21:25:21,009][44958] Updated weights for policy 0, policy_version 34310 (0.0009) [2023-10-12 21:25:21,375][44958] Updated weights for policy 0, policy_version 34320 (0.0009) [2023-10-12 21:25:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70451200. Throughput: 0: 1632.6, 1: 1637.2. Samples: 17623654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:25:21,443][43579] Avg episode reward: [(0, '270.130'), (1, '266.500')] [2023-10-12 21:25:21,525][44959] Updated weights for policy 1, policy_version 34500 (0.0008) [2023-10-12 21:25:21,744][44958] Updated weights for policy 0, policy_version 34330 (0.0008) [2023-10-12 21:25:21,905][44959] Updated weights for policy 1, policy_version 34510 (0.0008) [2023-10-12 21:25:22,263][44959] Updated weights for policy 1, policy_version 34520 (0.0010) [2023-10-12 21:25:25,997][44958] Updated weights for policy 0, policy_version 34340 (0.0009) [2023-10-12 21:25:26,316][44959] Updated weights for policy 1, policy_version 34530 (0.0008) [2023-10-12 21:25:26,370][44958] Updated weights for policy 0, policy_version 34350 (0.0009) [2023-10-12 21:25:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70516736. Throughput: 0: 1634.0, 1: 1636.4. Samples: 17643822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:25:26,443][43579] Avg episode reward: [(0, '264.670'), (1, '265.270')] [2023-10-12 21:25:26,687][44959] Updated weights for policy 1, policy_version 34540 (0.0009) [2023-10-12 21:25:26,747][44958] Updated weights for policy 0, policy_version 34360 (0.0007) [2023-10-12 21:25:27,051][44959] Updated weights for policy 1, policy_version 34550 (0.0007) [2023-10-12 21:25:27,422][44959] Updated weights for policy 1, policy_version 34560 (0.0008) [2023-10-12 21:25:30,918][44958] Updated weights for policy 0, policy_version 34370 (0.0007) [2023-10-12 21:25:31,290][44958] Updated weights for policy 0, policy_version 34380 (0.0009) [2023-10-12 21:25:31,443][43579] Fps is (10 sec: 13106.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70582272. Throughput: 0: 1643.4, 1: 1640.8. Samples: 17664090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:25:31,444][43579] Avg episode reward: [(0, '263.790'), (1, '262.270')] [2023-10-12 21:25:31,472][44959] Updated weights for policy 1, policy_version 34570 (0.0007) [2023-10-12 21:25:31,659][44958] Updated weights for policy 0, policy_version 34390 (0.0008) [2023-10-12 21:25:31,844][44959] Updated weights for policy 1, policy_version 34580 (0.0007) [2023-10-12 21:25:32,033][44958] Updated weights for policy 0, policy_version 34400 (0.0008) [2023-10-12 21:25:32,217][44959] Updated weights for policy 1, policy_version 34590 (0.0008) [2023-10-12 21:25:36,178][44958] Updated weights for policy 0, policy_version 34410 (0.0009) [2023-10-12 21:25:36,269][44959] Updated weights for policy 1, policy_version 34600 (0.0009) [2023-10-12 21:25:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70647808. Throughput: 0: 1639.9, 1: 1643.1. Samples: 17673334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:25:36,443][43579] Avg episode reward: [(0, '257.960'), (1, '274.940')] [2023-10-12 21:25:36,543][44958] Updated weights for policy 0, policy_version 34420 (0.0007) [2023-10-12 21:25:36,633][44959] Updated weights for policy 1, policy_version 34610 (0.0008) [2023-10-12 21:25:36,921][44958] Updated weights for policy 0, policy_version 34430 (0.0007) [2023-10-12 21:25:36,996][44959] Updated weights for policy 1, policy_version 34620 (0.0010) [2023-10-12 21:25:41,114][44958] Updated weights for policy 0, policy_version 34440 (0.0007) [2023-10-12 21:25:41,293][44959] Updated weights for policy 1, policy_version 34630 (0.0008) [2023-10-12 21:25:41,442][43579] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70713344. Throughput: 0: 1636.8, 1: 1640.5. Samples: 17693146. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-12 21:25:41,443][43579] Avg episode reward: [(0, '263.380'), (1, '274.690')] [2023-10-12 21:25:41,485][44958] Updated weights for policy 0, policy_version 34450 (0.0007) [2023-10-12 21:25:41,663][44959] Updated weights for policy 1, policy_version 34640 (0.0008) [2023-10-12 21:25:41,859][44958] Updated weights for policy 0, policy_version 34460 (0.0007) [2023-10-12 21:25:42,032][44959] Updated weights for policy 1, policy_version 34650 (0.0008) [2023-10-12 21:25:46,044][44958] Updated weights for policy 0, policy_version 34470 (0.0009) [2023-10-12 21:25:46,051][44959] Updated weights for policy 1, policy_version 34660 (0.0008) [2023-10-12 21:25:46,416][44958] Updated weights for policy 0, policy_version 34480 (0.0010) [2023-10-12 21:25:46,420][44959] Updated weights for policy 1, policy_version 34670 (0.0010) [2023-10-12 21:25:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70778880. Throughput: 0: 1634.4, 1: 1646.9. Samples: 17713192. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-12 21:25:46,443][43579] Avg episode reward: [(0, '265.610'), (1, '280.490')] [2023-10-12 21:25:46,786][44959] Updated weights for policy 1, policy_version 34680 (0.0008) [2023-10-12 21:25:46,800][44958] Updated weights for policy 0, policy_version 34490 (0.0009) [2023-10-12 21:25:47,012][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000034496_35323904.pth... [2023-10-12 21:25:47,047][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000032960_33751040.pth [2023-10-12 21:25:47,072][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000034688_35520512.pth... [2023-10-12 21:25:47,110][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000033120_33914880.pth [2023-10-12 21:25:51,035][44959] Updated weights for policy 1, policy_version 34690 (0.0008) [2023-10-12 21:25:51,134][44958] Updated weights for policy 0, policy_version 34500 (0.0008) [2023-10-12 21:25:51,391][44959] Updated weights for policy 1, policy_version 34700 (0.0008) [2023-10-12 21:25:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70844416. Throughput: 0: 1628.9, 1: 1647.1. Samples: 17722190. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-12 21:25:51,443][43579] Avg episode reward: [(0, '270.240'), (1, '277.910')] [2023-10-12 21:25:51,502][44958] Updated weights for policy 0, policy_version 34510 (0.0007) [2023-10-12 21:25:51,759][44959] Updated weights for policy 1, policy_version 34710 (0.0010) [2023-10-12 21:25:51,884][44958] Updated weights for policy 0, policy_version 34520 (0.0008) [2023-10-12 21:25:52,132][44959] Updated weights for policy 1, policy_version 34720 (0.0007) [2023-10-12 21:25:56,008][44958] Updated weights for policy 0, policy_version 34530 (0.0007) [2023-10-12 21:25:56,387][44958] Updated weights for policy 0, policy_version 34540 (0.0007) [2023-10-12 21:25:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70909952. Throughput: 0: 1633.3, 1: 1645.7. Samples: 17742188. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-12 21:25:56,443][43579] Avg episode reward: [(0, '272.650'), (1, '279.680')] [2023-10-12 21:25:56,563][44959] Updated weights for policy 1, policy_version 34730 (0.0008) [2023-10-12 21:25:56,756][44958] Updated weights for policy 0, policy_version 34550 (0.0007) [2023-10-12 21:25:56,936][44959] Updated weights for policy 1, policy_version 34740 (0.0009) [2023-10-12 21:25:57,128][44958] Updated weights for policy 0, policy_version 34560 (0.0008) [2023-10-12 21:25:57,306][44959] Updated weights for policy 1, policy_version 34750 (0.0009) [2023-10-12 21:26:01,250][44958] Updated weights for policy 0, policy_version 34570 (0.0008) [2023-10-12 21:26:01,396][44959] Updated weights for policy 1, policy_version 34760 (0.0008) [2023-10-12 21:26:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 70975488. Throughput: 0: 1631.9, 1: 1643.5. Samples: 17761940. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-12 21:26:01,443][43579] Avg episode reward: [(0, '266.040'), (1, '283.710')] [2023-10-12 21:26:01,625][44958] Updated weights for policy 0, policy_version 34580 (0.0007) [2023-10-12 21:26:01,766][44959] Updated weights for policy 1, policy_version 34770 (0.0009) [2023-10-12 21:26:01,996][44958] Updated weights for policy 0, policy_version 34590 (0.0008) [2023-10-12 21:26:02,129][44959] Updated weights for policy 1, policy_version 34780 (0.0009) [2023-10-12 21:26:06,265][44959] Updated weights for policy 1, policy_version 34790 (0.0010) [2023-10-12 21:26:06,396][44958] Updated weights for policy 0, policy_version 34600 (0.0008) [2023-10-12 21:26:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71041024. Throughput: 0: 1630.4, 1: 1643.8. Samples: 17770994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:06,444][43579] Avg episode reward: [(0, '269.410'), (1, '283.320')] [2023-10-12 21:26:06,638][44959] Updated weights for policy 1, policy_version 34800 (0.0009) [2023-10-12 21:26:06,776][44958] Updated weights for policy 0, policy_version 34610 (0.0007) [2023-10-12 21:26:07,008][44959] Updated weights for policy 1, policy_version 34810 (0.0007) [2023-10-12 21:26:07,146][44958] Updated weights for policy 0, policy_version 34620 (0.0008) [2023-10-12 21:26:11,242][44959] Updated weights for policy 1, policy_version 34820 (0.0008) [2023-10-12 21:26:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71106560. Throughput: 0: 1628.5, 1: 1645.6. Samples: 17791160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:11,443][43579] Avg episode reward: [(0, '263.530'), (1, '285.860')] [2023-10-12 21:26:11,501][44958] Updated weights for policy 0, policy_version 34630 (0.0011) [2023-10-12 21:26:11,604][44959] Updated weights for policy 1, policy_version 34830 (0.0008) [2023-10-12 21:26:11,870][44958] Updated weights for policy 0, policy_version 34640 (0.0008) [2023-10-12 21:26:11,975][44959] Updated weights for policy 1, policy_version 34840 (0.0009) [2023-10-12 21:26:12,244][44958] Updated weights for policy 0, policy_version 34650 (0.0009) [2023-10-12 21:26:15,946][44959] Updated weights for policy 1, policy_version 34850 (0.0008) [2023-10-12 21:26:16,261][44958] Updated weights for policy 0, policy_version 34660 (0.0010) [2023-10-12 21:26:16,303][44959] Updated weights for policy 1, policy_version 34860 (0.0010) [2023-10-12 21:26:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71172096. Throughput: 0: 1630.9, 1: 1640.1. Samples: 17811286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:16,443][43579] Avg episode reward: [(0, '261.620'), (1, '284.880')] [2023-10-12 21:26:16,624][44958] Updated weights for policy 0, policy_version 34670 (0.0008) [2023-10-12 21:26:16,669][44959] Updated weights for policy 1, policy_version 34870 (0.0008) [2023-10-12 21:26:17,002][44958] Updated weights for policy 0, policy_version 34680 (0.0008) [2023-10-12 21:26:17,039][44959] Updated weights for policy 1, policy_version 34880 (0.0009) [2023-10-12 21:26:21,262][44959] Updated weights for policy 1, policy_version 34890 (0.0008) [2023-10-12 21:26:21,304][44958] Updated weights for policy 0, policy_version 34690 (0.0009) [2023-10-12 21:26:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71237632. Throughput: 0: 1622.2, 1: 1641.0. Samples: 17820180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:21,443][43579] Avg episode reward: [(0, '260.960'), (1, '285.750')] [2023-10-12 21:26:21,633][44959] Updated weights for policy 1, policy_version 34900 (0.0007) [2023-10-12 21:26:21,676][44958] Updated weights for policy 0, policy_version 34700 (0.0007) [2023-10-12 21:26:22,009][44959] Updated weights for policy 1, policy_version 34910 (0.0008) [2023-10-12 21:26:22,055][44958] Updated weights for policy 0, policy_version 34710 (0.0007) [2023-10-12 21:26:22,420][44958] Updated weights for policy 0, policy_version 34720 (0.0010) [2023-10-12 21:26:26,153][44959] Updated weights for policy 1, policy_version 34920 (0.0008) [2023-10-12 21:26:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71303168. Throughput: 0: 1632.0, 1: 1647.1. Samples: 17840704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:26,443][43579] Avg episode reward: [(0, '263.120'), (1, '284.640')] [2023-10-12 21:26:26,466][44958] Updated weights for policy 0, policy_version 34730 (0.0009) [2023-10-12 21:26:26,519][44959] Updated weights for policy 1, policy_version 34930 (0.0008) [2023-10-12 21:26:26,829][44958] Updated weights for policy 0, policy_version 34740 (0.0009) [2023-10-12 21:26:26,881][44959] Updated weights for policy 1, policy_version 34940 (0.0008) [2023-10-12 21:26:27,204][44958] Updated weights for policy 0, policy_version 34750 (0.0007) [2023-10-12 21:26:31,043][44959] Updated weights for policy 1, policy_version 34950 (0.0009) [2023-10-12 21:26:31,382][44958] Updated weights for policy 0, policy_version 34760 (0.0008) [2023-10-12 21:26:31,400][44959] Updated weights for policy 1, policy_version 34960 (0.0007) [2023-10-12 21:26:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 71368704. Throughput: 0: 1635.6, 1: 1638.6. Samples: 17860530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:31,443][43579] Avg episode reward: [(0, '266.760'), (1, '282.610')] [2023-10-12 21:26:31,762][44958] Updated weights for policy 0, policy_version 34770 (0.0007) [2023-10-12 21:26:31,764][44959] Updated weights for policy 1, policy_version 34970 (0.0007) [2023-10-12 21:26:32,136][44958] Updated weights for policy 0, policy_version 34780 (0.0009) [2023-10-12 21:26:35,999][44959] Updated weights for policy 1, policy_version 34980 (0.0009) [2023-10-12 21:26:36,078][44958] Updated weights for policy 0, policy_version 34790 (0.0010) [2023-10-12 21:26:36,368][44959] Updated weights for policy 1, policy_version 34990 (0.0008) [2023-10-12 21:26:36,440][44958] Updated weights for policy 0, policy_version 34800 (0.0008) [2023-10-12 21:26:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71434240. Throughput: 0: 1634.2, 1: 1643.6. Samples: 17869692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:36,443][43579] Avg episode reward: [(0, '266.160'), (1, '280.750')] [2023-10-12 21:26:36,737][44959] Updated weights for policy 1, policy_version 35000 (0.0009) [2023-10-12 21:26:36,807][44958] Updated weights for policy 0, policy_version 34810 (0.0009) [2023-10-12 21:26:41,171][44959] Updated weights for policy 1, policy_version 35010 (0.0009) [2023-10-12 21:26:41,284][44958] Updated weights for policy 0, policy_version 34820 (0.0009) [2023-10-12 21:26:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71499776. Throughput: 0: 1638.4, 1: 1644.7. Samples: 17889928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:41,443][43579] Avg episode reward: [(0, '266.680'), (1, '280.190')] [2023-10-12 21:26:41,587][44959] Updated weights for policy 1, policy_version 35020 (0.0008) [2023-10-12 21:26:41,656][44958] Updated weights for policy 0, policy_version 34830 (0.0009) [2023-10-12 21:26:41,947][44959] Updated weights for policy 1, policy_version 35030 (0.0008) [2023-10-12 21:26:42,019][44958] Updated weights for policy 0, policy_version 34840 (0.0009) [2023-10-12 21:26:42,327][44959] Updated weights for policy 1, policy_version 35040 (0.0008) [2023-10-12 21:26:46,347][44958] Updated weights for policy 0, policy_version 34850 (0.0008) [2023-10-12 21:26:46,421][44959] Updated weights for policy 1, policy_version 35050 (0.0009) [2023-10-12 21:26:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71565312. Throughput: 0: 1638.4, 1: 1646.9. Samples: 17909776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:46,443][43579] Avg episode reward: [(0, '267.790'), (1, '276.900')] [2023-10-12 21:26:46,737][44958] Updated weights for policy 0, policy_version 34860 (0.0009) [2023-10-12 21:26:46,783][44959] Updated weights for policy 1, policy_version 35060 (0.0009) [2023-10-12 21:26:47,111][44958] Updated weights for policy 0, policy_version 34870 (0.0007) [2023-10-12 21:26:47,148][44959] Updated weights for policy 1, policy_version 35070 (0.0007) [2023-10-12 21:26:47,482][44958] Updated weights for policy 0, policy_version 34880 (0.0008) [2023-10-12 21:26:51,308][44959] Updated weights for policy 1, policy_version 35080 (0.0007) [2023-10-12 21:26:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71630848. Throughput: 0: 1633.1, 1: 1645.7. Samples: 17918538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:26:51,443][43579] Avg episode reward: [(0, '264.820'), (1, '273.220')] [2023-10-12 21:26:51,593][44958] Updated weights for policy 0, policy_version 34890 (0.0009) [2023-10-12 21:26:51,678][44959] Updated weights for policy 1, policy_version 35090 (0.0009) [2023-10-12 21:26:51,968][44958] Updated weights for policy 0, policy_version 34900 (0.0009) [2023-10-12 21:26:52,035][44959] Updated weights for policy 1, policy_version 35100 (0.0009) [2023-10-12 21:26:52,336][44958] Updated weights for policy 0, policy_version 34910 (0.0009) [2023-10-12 21:26:56,180][44959] Updated weights for policy 1, policy_version 35110 (0.0008) [2023-10-12 21:26:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71696384. Throughput: 0: 1638.3, 1: 1643.8. Samples: 17938852. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-10-12 21:26:56,444][43579] Avg episode reward: [(0, '260.130'), (1, '273.440')] [2023-10-12 21:26:56,502][44958] Updated weights for policy 0, policy_version 34920 (0.0008) [2023-10-12 21:26:56,548][44959] Updated weights for policy 1, policy_version 35120 (0.0008) [2023-10-12 21:26:56,873][44958] Updated weights for policy 0, policy_version 34930 (0.0007) [2023-10-12 21:26:56,921][44959] Updated weights for policy 1, policy_version 35130 (0.0007) [2023-10-12 21:26:57,241][44958] Updated weights for policy 0, policy_version 34940 (0.0008) [2023-10-12 21:27:00,929][44959] Updated weights for policy 1, policy_version 35140 (0.0009) [2023-10-12 21:27:01,296][44959] Updated weights for policy 1, policy_version 35150 (0.0008) [2023-10-12 21:27:01,331][44958] Updated weights for policy 0, policy_version 34950 (0.0009) [2023-10-12 21:27:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71761920. Throughput: 0: 1638.2, 1: 1638.2. Samples: 17958726. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-10-12 21:27:01,443][43579] Avg episode reward: [(0, '263.590'), (1, '272.610')] [2023-10-12 21:27:01,653][44959] Updated weights for policy 1, policy_version 35160 (0.0009) [2023-10-12 21:27:01,699][44958] Updated weights for policy 0, policy_version 34960 (0.0008) [2023-10-12 21:27:02,065][44958] Updated weights for policy 0, policy_version 34970 (0.0008) [2023-10-12 21:27:05,812][44959] Updated weights for policy 1, policy_version 35170 (0.0008) [2023-10-12 21:27:06,119][44958] Updated weights for policy 0, policy_version 34980 (0.0009) [2023-10-12 21:27:06,180][44959] Updated weights for policy 1, policy_version 35180 (0.0007) [2023-10-12 21:27:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71827456. Throughput: 0: 1643.2, 1: 1645.7. Samples: 17968180. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-10-12 21:27:06,443][43579] Avg episode reward: [(0, '265.270'), (1, '272.160')] [2023-10-12 21:27:06,496][44958] Updated weights for policy 0, policy_version 34990 (0.0008) [2023-10-12 21:27:06,558][44959] Updated weights for policy 1, policy_version 35190 (0.0008) [2023-10-12 21:27:06,861][44958] Updated weights for policy 0, policy_version 35000 (0.0010) [2023-10-12 21:27:06,924][44959] Updated weights for policy 1, policy_version 35200 (0.0008) [2023-10-12 21:27:11,127][44959] Updated weights for policy 1, policy_version 35210 (0.0009) [2023-10-12 21:27:11,395][44958] Updated weights for policy 0, policy_version 35010 (0.0012) [2023-10-12 21:27:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71892992. Throughput: 0: 1632.2, 1: 1643.3. Samples: 17988104. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-10-12 21:27:11,443][43579] Avg episode reward: [(0, '268.910'), (1, '279.440')] [2023-10-12 21:27:11,499][44959] Updated weights for policy 1, policy_version 35220 (0.0008) [2023-10-12 21:27:11,765][44958] Updated weights for policy 0, policy_version 35020 (0.0007) [2023-10-12 21:27:11,867][44959] Updated weights for policy 1, policy_version 35230 (0.0009) [2023-10-12 21:27:12,130][44958] Updated weights for policy 0, policy_version 35030 (0.0008) [2023-10-12 21:27:12,499][44958] Updated weights for policy 0, policy_version 35040 (0.0008) [2023-10-12 21:27:16,187][44959] Updated weights for policy 1, policy_version 35240 (0.0007) [2023-10-12 21:27:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 71958528. Throughput: 0: 1633.8, 1: 1644.0. Samples: 18008030. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-10-12 21:27:16,443][43579] Avg episode reward: [(0, '268.400'), (1, '274.610')] [2023-10-12 21:27:16,561][44959] Updated weights for policy 1, policy_version 35250 (0.0008) [2023-10-12 21:27:16,647][44958] Updated weights for policy 0, policy_version 35050 (0.0010) [2023-10-12 21:27:16,928][44959] Updated weights for policy 1, policy_version 35260 (0.0008) [2023-10-12 21:27:17,014][44958] Updated weights for policy 0, policy_version 35060 (0.0009) [2023-10-12 21:27:17,392][44958] Updated weights for policy 0, policy_version 35070 (0.0009) [2023-10-12 21:27:21,018][44959] Updated weights for policy 1, policy_version 35270 (0.0008) [2023-10-12 21:27:21,391][44959] Updated weights for policy 1, policy_version 35280 (0.0007) [2023-10-12 21:27:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72024064. Throughput: 0: 1635.4, 1: 1644.7. Samples: 18017296. Policy #0 lag: (min: 12.0, avg: 14.2, max: 44.0) [2023-10-12 21:27:21,443][43579] Avg episode reward: [(0, '274.340'), (1, '274.210')] [2023-10-12 21:27:21,534][44958] Updated weights for policy 0, policy_version 35080 (0.0008) [2023-10-12 21:27:21,760][44959] Updated weights for policy 1, policy_version 35290 (0.0008) [2023-10-12 21:27:21,905][44958] Updated weights for policy 0, policy_version 35090 (0.0008) [2023-10-12 21:27:22,269][44958] Updated weights for policy 0, policy_version 35100 (0.0007) [2023-10-12 21:27:26,189][44959] Updated weights for policy 1, policy_version 35300 (0.0009) [2023-10-12 21:27:26,317][44958] Updated weights for policy 0, policy_version 35110 (0.0008) [2023-10-12 21:27:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72089600. Throughput: 0: 1632.8, 1: 1645.0. Samples: 18037430. Policy #0 lag: (min: 12.0, avg: 14.2, max: 44.0) [2023-10-12 21:27:26,444][43579] Avg episode reward: [(0, '278.600'), (1, '270.810')] [2023-10-12 21:27:26,569][44959] Updated weights for policy 1, policy_version 35310 (0.0008) [2023-10-12 21:27:26,696][44958] Updated weights for policy 0, policy_version 35120 (0.0007) [2023-10-12 21:27:26,942][44959] Updated weights for policy 1, policy_version 35320 (0.0007) [2023-10-12 21:27:27,060][44958] Updated weights for policy 0, policy_version 35130 (0.0007) [2023-10-12 21:27:30,987][44959] Updated weights for policy 1, policy_version 35330 (0.0009) [2023-10-12 21:27:31,352][44958] Updated weights for policy 0, policy_version 35140 (0.0007) [2023-10-12 21:27:31,364][44959] Updated weights for policy 1, policy_version 35340 (0.0007) [2023-10-12 21:27:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72155136. Throughput: 0: 1639.8, 1: 1640.6. Samples: 18057394. Policy #0 lag: (min: 12.0, avg: 14.2, max: 44.0) [2023-10-12 21:27:31,443][43579] Avg episode reward: [(0, '277.500'), (1, '269.140')] [2023-10-12 21:27:31,729][44958] Updated weights for policy 0, policy_version 35150 (0.0008) [2023-10-12 21:27:31,729][44959] Updated weights for policy 1, policy_version 35350 (0.0007) [2023-10-12 21:27:32,094][44959] Updated weights for policy 1, policy_version 35360 (0.0008) [2023-10-12 21:27:32,098][44958] Updated weights for policy 0, policy_version 35160 (0.0007) [2023-10-12 21:27:36,247][44959] Updated weights for policy 1, policy_version 35370 (0.0010) [2023-10-12 21:27:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72220672. Throughput: 0: 1641.5, 1: 1642.8. Samples: 18066332. Policy #0 lag: (min: 12.0, avg: 14.2, max: 44.0) [2023-10-12 21:27:36,443][43579] Avg episode reward: [(0, '277.510'), (1, '270.760')] [2023-10-12 21:27:36,510][44958] Updated weights for policy 0, policy_version 35170 (0.0009) [2023-10-12 21:27:36,616][44959] Updated weights for policy 1, policy_version 35380 (0.0008) [2023-10-12 21:27:36,910][44958] Updated weights for policy 0, policy_version 35180 (0.0007) [2023-10-12 21:27:36,983][44959] Updated weights for policy 1, policy_version 35390 (0.0008) [2023-10-12 21:27:37,281][44958] Updated weights for policy 0, policy_version 35190 (0.0009) [2023-10-12 21:27:37,657][44958] Updated weights for policy 0, policy_version 35200 (0.0007) [2023-10-12 21:27:41,144][44959] Updated weights for policy 1, policy_version 35400 (0.0008) [2023-10-12 21:27:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72286208. Throughput: 0: 1634.1, 1: 1644.4. Samples: 18086382. Policy #0 lag: (min: 12.0, avg: 14.2, max: 44.0) [2023-10-12 21:27:41,443][43579] Avg episode reward: [(0, '268.590'), (1, '270.080')] [2023-10-12 21:27:41,515][44959] Updated weights for policy 1, policy_version 35410 (0.0007) [2023-10-12 21:27:41,769][44958] Updated weights for policy 0, policy_version 35210 (0.0007) [2023-10-12 21:27:41,877][44959] Updated weights for policy 1, policy_version 35420 (0.0007) [2023-10-12 21:27:42,144][44958] Updated weights for policy 0, policy_version 35220 (0.0009) [2023-10-12 21:27:42,517][44958] Updated weights for policy 0, policy_version 35230 (0.0008) [2023-10-12 21:27:45,924][44959] Updated weights for policy 1, policy_version 35430 (0.0007) [2023-10-12 21:27:46,289][44959] Updated weights for policy 1, policy_version 35440 (0.0008) [2023-10-12 21:27:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72351744. Throughput: 0: 1636.5, 1: 1647.6. Samples: 18106510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:27:46,443][43579] Avg episode reward: [(0, '267.760'), (1, '274.480')] [2023-10-12 21:27:46,593][44958] Updated weights for policy 0, policy_version 35240 (0.0008) [2023-10-12 21:27:46,663][44959] Updated weights for policy 1, policy_version 35450 (0.0008) [2023-10-12 21:27:46,874][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000035456_36306944.pth... [2023-10-12 21:27:46,917][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000033888_34701312.pth [2023-10-12 21:27:46,968][44958] Updated weights for policy 0, policy_version 35250 (0.0008) [2023-10-12 21:27:47,344][44958] Updated weights for policy 0, policy_version 35260 (0.0007) [2023-10-12 21:27:47,490][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000035264_36110336.pth... [2023-10-12 21:27:47,528][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000033728_34537472.pth [2023-10-12 21:27:51,072][44959] Updated weights for policy 1, policy_version 35460 (0.0010) [2023-10-12 21:27:51,438][44959] Updated weights for policy 1, policy_version 35470 (0.0010) [2023-10-12 21:27:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72417280. Throughput: 0: 1631.2, 1: 1643.0. Samples: 18115518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:27:51,443][43579] Avg episode reward: [(0, '268.300'), (1, '277.840')] [2023-10-12 21:27:51,545][44958] Updated weights for policy 0, policy_version 35270 (0.0007) [2023-10-12 21:27:51,799][44959] Updated weights for policy 1, policy_version 35480 (0.0007) [2023-10-12 21:27:51,911][44958] Updated weights for policy 0, policy_version 35280 (0.0008) [2023-10-12 21:27:52,274][44958] Updated weights for policy 0, policy_version 35290 (0.0008) [2023-10-12 21:27:55,868][44959] Updated weights for policy 1, policy_version 35490 (0.0009) [2023-10-12 21:27:56,243][44959] Updated weights for policy 1, policy_version 35500 (0.0009) [2023-10-12 21:27:56,320][44958] Updated weights for policy 0, policy_version 35300 (0.0009) [2023-10-12 21:27:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72482816. Throughput: 0: 1641.5, 1: 1639.4. Samples: 18135744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:27:56,443][43579] Avg episode reward: [(0, '268.210'), (1, '283.370')] [2023-10-12 21:27:56,601][44959] Updated weights for policy 1, policy_version 35510 (0.0008) [2023-10-12 21:27:56,695][44958] Updated weights for policy 0, policy_version 35310 (0.0008) [2023-10-12 21:27:56,970][44959] Updated weights for policy 1, policy_version 35520 (0.0008) [2023-10-12 21:27:57,071][44958] Updated weights for policy 0, policy_version 35320 (0.0009) [2023-10-12 21:28:01,034][44958] Updated weights for policy 0, policy_version 35330 (0.0010) [2023-10-12 21:28:01,114][44959] Updated weights for policy 1, policy_version 35530 (0.0010) [2023-10-12 21:28:01,416][44958] Updated weights for policy 0, policy_version 35340 (0.0008) [2023-10-12 21:28:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72548352. Throughput: 0: 1640.4, 1: 1636.3. Samples: 18155478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:28:01,443][43579] Avg episode reward: [(0, '264.170'), (1, '280.500')] [2023-10-12 21:28:01,482][44959] Updated weights for policy 1, policy_version 35540 (0.0010) [2023-10-12 21:28:01,772][44958] Updated weights for policy 0, policy_version 35350 (0.0009) [2023-10-12 21:28:01,850][44959] Updated weights for policy 1, policy_version 35550 (0.0009) [2023-10-12 21:28:02,140][44958] Updated weights for policy 0, policy_version 35360 (0.0009) [2023-10-12 21:28:05,812][44959] Updated weights for policy 1, policy_version 35560 (0.0008) [2023-10-12 21:28:06,173][44959] Updated weights for policy 1, policy_version 35570 (0.0007) [2023-10-12 21:28:06,303][44958] Updated weights for policy 0, policy_version 35370 (0.0009) [2023-10-12 21:28:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72613888. Throughput: 0: 1641.0, 1: 1641.3. Samples: 18165002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:28:06,443][43579] Avg episode reward: [(0, '265.490'), (1, '279.930')] [2023-10-12 21:28:06,545][44959] Updated weights for policy 1, policy_version 35580 (0.0009) [2023-10-12 21:28:06,673][44958] Updated weights for policy 0, policy_version 35380 (0.0010) [2023-10-12 21:28:07,045][44958] Updated weights for policy 0, policy_version 35390 (0.0008) [2023-10-12 21:28:10,931][44959] Updated weights for policy 1, policy_version 35590 (0.0008) [2023-10-12 21:28:11,257][44958] Updated weights for policy 0, policy_version 35400 (0.0007) [2023-10-12 21:28:11,319][44959] Updated weights for policy 1, policy_version 35600 (0.0008) [2023-10-12 21:28:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72679424. Throughput: 0: 1639.9, 1: 1644.2. Samples: 18185214. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-12 21:28:11,443][43579] Avg episode reward: [(0, '271.100'), (1, '281.110')] [2023-10-12 21:28:11,625][44958] Updated weights for policy 0, policy_version 35410 (0.0008) [2023-10-12 21:28:11,690][44959] Updated weights for policy 1, policy_version 35610 (0.0008) [2023-10-12 21:28:11,990][44958] Updated weights for policy 0, policy_version 35420 (0.0009) [2023-10-12 21:28:15,825][44959] Updated weights for policy 1, policy_version 35620 (0.0008) [2023-10-12 21:28:16,089][44958] Updated weights for policy 0, policy_version 35430 (0.0007) [2023-10-12 21:28:16,190][44959] Updated weights for policy 1, policy_version 35630 (0.0008) [2023-10-12 21:28:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72744960. Throughput: 0: 1632.9, 1: 1641.6. Samples: 18204748. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-12 21:28:16,443][43579] Avg episode reward: [(0, '272.500'), (1, '282.110')] [2023-10-12 21:28:16,452][44958] Updated weights for policy 0, policy_version 35440 (0.0010) [2023-10-12 21:28:16,556][44959] Updated weights for policy 1, policy_version 35640 (0.0007) [2023-10-12 21:28:16,831][44958] Updated weights for policy 0, policy_version 35450 (0.0008) [2023-10-12 21:28:20,731][44959] Updated weights for policy 1, policy_version 35650 (0.0008) [2023-10-12 21:28:21,110][44959] Updated weights for policy 1, policy_version 35660 (0.0010) [2023-10-12 21:28:21,119][44958] Updated weights for policy 0, policy_version 35460 (0.0009) [2023-10-12 21:28:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72810496. Throughput: 0: 1638.1, 1: 1647.8. Samples: 18214200. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-12 21:28:21,443][43579] Avg episode reward: [(0, '270.230'), (1, '279.050')] [2023-10-12 21:28:21,470][44959] Updated weights for policy 1, policy_version 35670 (0.0009) [2023-10-12 21:28:21,494][44958] Updated weights for policy 0, policy_version 35470 (0.0008) [2023-10-12 21:28:21,843][44959] Updated weights for policy 1, policy_version 35680 (0.0008) [2023-10-12 21:28:21,866][44958] Updated weights for policy 0, policy_version 35480 (0.0007) [2023-10-12 21:28:25,913][44959] Updated weights for policy 1, policy_version 35690 (0.0008) [2023-10-12 21:28:26,154][44958] Updated weights for policy 0, policy_version 35490 (0.0008) [2023-10-12 21:28:26,271][44959] Updated weights for policy 1, policy_version 35700 (0.0008) [2023-10-12 21:28:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 72876032. Throughput: 0: 1637.5, 1: 1645.5. Samples: 18234116. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-12 21:28:26,443][43579] Avg episode reward: [(0, '269.680'), (1, '273.700')] [2023-10-12 21:28:26,548][44958] Updated weights for policy 0, policy_version 35500 (0.0009) [2023-10-12 21:28:26,647][44959] Updated weights for policy 1, policy_version 35710 (0.0007) [2023-10-12 21:28:26,924][44958] Updated weights for policy 0, policy_version 35510 (0.0009) [2023-10-12 21:28:27,288][44958] Updated weights for policy 0, policy_version 35520 (0.0011) [2023-10-12 21:28:30,657][44959] Updated weights for policy 1, policy_version 35720 (0.0008) [2023-10-12 21:28:31,030][44959] Updated weights for policy 1, policy_version 35730 (0.0010) [2023-10-12 21:28:31,404][44959] Updated weights for policy 1, policy_version 35740 (0.0008) [2023-10-12 21:28:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 72941568. Throughput: 0: 1632.0, 1: 1639.5. Samples: 18253728. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-12 21:28:31,444][43579] Avg episode reward: [(0, '270.530'), (1, '277.190')] [2023-10-12 21:28:31,592][44958] Updated weights for policy 0, policy_version 35530 (0.0010) [2023-10-12 21:28:31,967][44958] Updated weights for policy 0, policy_version 35540 (0.0007) [2023-10-12 21:28:32,338][44958] Updated weights for policy 0, policy_version 35550 (0.0010) [2023-10-12 21:28:35,666][44959] Updated weights for policy 1, policy_version 35750 (0.0009) [2023-10-12 21:28:36,042][44959] Updated weights for policy 1, policy_version 35760 (0.0007) [2023-10-12 21:28:36,403][44959] Updated weights for policy 1, policy_version 35770 (0.0011) [2023-10-12 21:28:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73007104. Throughput: 0: 1632.5, 1: 1646.8. Samples: 18263090. Policy #0 lag: (min: 6.0, avg: 9.1, max: 38.0) [2023-10-12 21:28:36,443][43579] Avg episode reward: [(0, '270.210'), (1, '277.520')] [2023-10-12 21:28:36,615][44958] Updated weights for policy 0, policy_version 35560 (0.0010) [2023-10-12 21:28:36,989][44958] Updated weights for policy 0, policy_version 35570 (0.0010) [2023-10-12 21:28:37,356][44958] Updated weights for policy 0, policy_version 35580 (0.0011) [2023-10-12 21:28:40,628][44959] Updated weights for policy 1, policy_version 35780 (0.0009) [2023-10-12 21:28:40,988][44959] Updated weights for policy 1, policy_version 35790 (0.0007) [2023-10-12 21:28:41,357][44959] Updated weights for policy 1, policy_version 35800 (0.0007) [2023-10-12 21:28:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73072640. Throughput: 0: 1624.0, 1: 1650.1. Samples: 18283082. Policy #0 lag: (min: 6.0, avg: 9.1, max: 38.0) [2023-10-12 21:28:41,443][43579] Avg episode reward: [(0, '270.290'), (1, '278.690')] [2023-10-12 21:28:41,654][44958] Updated weights for policy 0, policy_version 35590 (0.0009) [2023-10-12 21:28:42,021][44958] Updated weights for policy 0, policy_version 35600 (0.0009) [2023-10-12 21:28:42,399][44958] Updated weights for policy 0, policy_version 35610 (0.0009) [2023-10-12 21:28:45,605][44959] Updated weights for policy 1, policy_version 35810 (0.0007) [2023-10-12 21:28:45,988][44959] Updated weights for policy 1, policy_version 35820 (0.0008) [2023-10-12 21:28:46,353][44959] Updated weights for policy 1, policy_version 35830 (0.0008) [2023-10-12 21:28:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73138176. Throughput: 0: 1633.3, 1: 1649.1. Samples: 18303186. Policy #0 lag: (min: 6.0, avg: 9.1, max: 38.0) [2023-10-12 21:28:46,443][43579] Avg episode reward: [(0, '267.920'), (1, '277.860')] [2023-10-12 21:28:46,505][44958] Updated weights for policy 0, policy_version 35620 (0.0009) [2023-10-12 21:28:46,721][44959] Updated weights for policy 1, policy_version 35840 (0.0008) [2023-10-12 21:28:46,881][44958] Updated weights for policy 0, policy_version 35630 (0.0008) [2023-10-12 21:28:47,252][44958] Updated weights for policy 0, policy_version 35640 (0.0010) [2023-10-12 21:28:50,719][44959] Updated weights for policy 1, policy_version 35850 (0.0008) [2023-10-12 21:28:51,097][44959] Updated weights for policy 1, policy_version 35860 (0.0008) [2023-10-12 21:28:51,428][44958] Updated weights for policy 0, policy_version 35650 (0.0010) [2023-10-12 21:28:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73203712. Throughput: 0: 1626.1, 1: 1648.9. Samples: 18312376. Policy #0 lag: (min: 6.0, avg: 9.1, max: 38.0) [2023-10-12 21:28:51,443][43579] Avg episode reward: [(0, '271.530'), (1, '279.120')] [2023-10-12 21:28:51,464][44959] Updated weights for policy 1, policy_version 35870 (0.0009) [2023-10-12 21:28:51,798][44958] Updated weights for policy 0, policy_version 35660 (0.0009) [2023-10-12 21:28:52,174][44958] Updated weights for policy 0, policy_version 35670 (0.0008) [2023-10-12 21:28:52,534][44958] Updated weights for policy 0, policy_version 35680 (0.0009) [2023-10-12 21:28:55,517][44959] Updated weights for policy 1, policy_version 35880 (0.0011) [2023-10-12 21:28:55,883][44959] Updated weights for policy 1, policy_version 35890 (0.0009) [2023-10-12 21:28:56,249][44959] Updated weights for policy 1, policy_version 35900 (0.0009) [2023-10-12 21:28:56,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 73302016. Throughput: 0: 1631.3, 1: 1652.6. Samples: 18332988. Policy #0 lag: (min: 6.0, avg: 9.1, max: 38.0) [2023-10-12 21:28:56,444][43579] Avg episode reward: [(0, '271.610'), (1, '276.960')] [2023-10-12 21:28:56,777][44958] Updated weights for policy 0, policy_version 35690 (0.0010) [2023-10-12 21:28:57,152][44958] Updated weights for policy 0, policy_version 35700 (0.0010) [2023-10-12 21:28:57,523][44958] Updated weights for policy 0, policy_version 35710 (0.0009) [2023-10-12 21:29:00,615][44959] Updated weights for policy 1, policy_version 35910 (0.0010) [2023-10-12 21:29:01,009][44959] Updated weights for policy 1, policy_version 35920 (0.0009) [2023-10-12 21:29:01,367][44959] Updated weights for policy 1, policy_version 35930 (0.0007) [2023-10-12 21:29:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73334784. Throughput: 0: 1637.4, 1: 1639.7. Samples: 18352218. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 21:29:01,443][43579] Avg episode reward: [(0, '270.170'), (1, '277.950')] [2023-10-12 21:29:01,598][44958] Updated weights for policy 0, policy_version 35720 (0.0007) [2023-10-12 21:29:01,976][44958] Updated weights for policy 0, policy_version 35730 (0.0007) [2023-10-12 21:29:02,353][44958] Updated weights for policy 0, policy_version 35740 (0.0009) [2023-10-12 21:29:05,436][44959] Updated weights for policy 1, policy_version 35940 (0.0007) [2023-10-12 21:29:05,812][44959] Updated weights for policy 1, policy_version 35950 (0.0009) [2023-10-12 21:29:06,189][44959] Updated weights for policy 1, policy_version 35960 (0.0009) [2023-10-12 21:29:06,442][43579] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73400320. Throughput: 0: 1631.0, 1: 1647.1. Samples: 18361712. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 21:29:06,443][43579] Avg episode reward: [(0, '271.680'), (1, '273.290')] [2023-10-12 21:29:06,513][44958] Updated weights for policy 0, policy_version 35750 (0.0009) [2023-10-12 21:29:06,883][44958] Updated weights for policy 0, policy_version 35760 (0.0008) [2023-10-12 21:29:07,257][44958] Updated weights for policy 0, policy_version 35770 (0.0010) [2023-10-12 21:29:10,467][44959] Updated weights for policy 1, policy_version 35970 (0.0010) [2023-10-12 21:29:10,848][44959] Updated weights for policy 1, policy_version 35980 (0.0009) [2023-10-12 21:29:11,214][44959] Updated weights for policy 1, policy_version 35990 (0.0007) [2023-10-12 21:29:11,423][44958] Updated weights for policy 0, policy_version 35780 (0.0009) [2023-10-12 21:29:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73465856. Throughput: 0: 1636.1, 1: 1651.3. Samples: 18382050. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 21:29:11,443][43579] Avg episode reward: [(0, '270.970'), (1, '269.800')] [2023-10-12 21:29:11,578][44959] Updated weights for policy 1, policy_version 36000 (0.0007) [2023-10-12 21:29:11,809][44958] Updated weights for policy 0, policy_version 35790 (0.0010) [2023-10-12 21:29:12,187][44958] Updated weights for policy 0, policy_version 35800 (0.0010) [2023-10-12 21:29:15,858][44959] Updated weights for policy 1, policy_version 36010 (0.0011) [2023-10-12 21:29:16,220][44959] Updated weights for policy 1, policy_version 36020 (0.0008) [2023-10-12 21:29:16,223][44958] Updated weights for policy 0, policy_version 35810 (0.0009) [2023-10-12 21:29:16,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 73531392. Throughput: 0: 1642.3, 1: 1645.5. Samples: 18401680. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 21:29:16,444][43579] Avg episode reward: [(0, '270.220'), (1, '267.570')] [2023-10-12 21:29:16,592][44959] Updated weights for policy 1, policy_version 36030 (0.0008) [2023-10-12 21:29:16,596][44958] Updated weights for policy 0, policy_version 35820 (0.0008) [2023-10-12 21:29:16,967][44958] Updated weights for policy 0, policy_version 35830 (0.0009) [2023-10-12 21:29:17,341][44958] Updated weights for policy 0, policy_version 35840 (0.0009) [2023-10-12 21:29:20,583][44959] Updated weights for policy 1, policy_version 36040 (0.0008) [2023-10-12 21:29:20,951][44959] Updated weights for policy 1, policy_version 36050 (0.0009) [2023-10-12 21:29:21,317][44959] Updated weights for policy 1, policy_version 36060 (0.0009) [2023-10-12 21:29:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73596928. Throughput: 0: 1642.9, 1: 1647.5. Samples: 18411158. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-12 21:29:21,444][43579] Avg episode reward: [(0, '271.960'), (1, '272.150')] [2023-10-12 21:29:21,458][44958] Updated weights for policy 0, policy_version 35850 (0.0009) [2023-10-12 21:29:21,837][44958] Updated weights for policy 0, policy_version 35860 (0.0009) [2023-10-12 21:29:22,212][44958] Updated weights for policy 0, policy_version 35870 (0.0007) [2023-10-12 21:29:25,365][44959] Updated weights for policy 1, policy_version 36070 (0.0008) [2023-10-12 21:29:25,728][44959] Updated weights for policy 1, policy_version 36080 (0.0009) [2023-10-12 21:29:26,091][44959] Updated weights for policy 1, policy_version 36090 (0.0009) [2023-10-12 21:29:26,315][44958] Updated weights for policy 0, policy_version 35880 (0.0009) [2023-10-12 21:29:26,443][43579] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 73695232. Throughput: 0: 1652.6, 1: 1648.5. Samples: 18431634. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-12 21:29:26,444][43579] Avg episode reward: [(0, '274.550'), (1, '269.620')] [2023-10-12 21:29:26,689][44958] Updated weights for policy 0, policy_version 35890 (0.0008) [2023-10-12 21:29:27,061][44958] Updated weights for policy 0, policy_version 35900 (0.0009) [2023-10-12 21:29:30,467][44959] Updated weights for policy 1, policy_version 36100 (0.0008) [2023-10-12 21:29:30,841][44959] Updated weights for policy 1, policy_version 36110 (0.0009) [2023-10-12 21:29:31,211][44959] Updated weights for policy 1, policy_version 36120 (0.0009) [2023-10-12 21:29:31,379][44958] Updated weights for policy 0, policy_version 35910 (0.0008) [2023-10-12 21:29:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 73728000. Throughput: 0: 1649.2, 1: 1645.7. Samples: 18451456. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-12 21:29:31,443][43579] Avg episode reward: [(0, '269.880'), (1, '267.110')] [2023-10-12 21:29:31,751][44958] Updated weights for policy 0, policy_version 35920 (0.0009) [2023-10-12 21:29:32,123][44958] Updated weights for policy 0, policy_version 35930 (0.0010) [2023-10-12 21:29:35,255][44959] Updated weights for policy 1, policy_version 36130 (0.0007) [2023-10-12 21:29:35,630][44959] Updated weights for policy 1, policy_version 36140 (0.0007) [2023-10-12 21:29:36,005][44959] Updated weights for policy 1, policy_version 36150 (0.0007) [2023-10-12 21:29:36,295][44958] Updated weights for policy 0, policy_version 35940 (0.0009) [2023-10-12 21:29:36,364][44959] Updated weights for policy 1, policy_version 36160 (0.0007) [2023-10-12 21:29:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 73826304. Throughput: 0: 1655.3, 1: 1650.2. Samples: 18461126. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-12 21:29:36,443][43579] Avg episode reward: [(0, '268.510'), (1, '266.340')] [2023-10-12 21:29:36,663][44958] Updated weights for policy 0, policy_version 35950 (0.0010) [2023-10-12 21:29:37,039][44958] Updated weights for policy 0, policy_version 35960 (0.0010) [2023-10-12 21:29:40,665][44959] Updated weights for policy 1, policy_version 36170 (0.0007) [2023-10-12 21:29:41,037][44959] Updated weights for policy 1, policy_version 36180 (0.0008) [2023-10-12 21:29:41,140][44958] Updated weights for policy 0, policy_version 35970 (0.0010) [2023-10-12 21:29:41,396][44959] Updated weights for policy 1, policy_version 36190 (0.0010) [2023-10-12 21:29:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73859072. Throughput: 0: 1652.0, 1: 1643.7. Samples: 18481292. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-12 21:29:41,443][43579] Avg episode reward: [(0, '268.770'), (1, '259.480')] [2023-10-12 21:29:41,509][44958] Updated weights for policy 0, policy_version 35980 (0.0009) [2023-10-12 21:29:41,880][44958] Updated weights for policy 0, policy_version 35990 (0.0010) [2023-10-12 21:29:42,250][44958] Updated weights for policy 0, policy_version 36000 (0.0007) [2023-10-12 21:29:45,574][44959] Updated weights for policy 1, policy_version 36200 (0.0008) [2023-10-12 21:29:45,957][44959] Updated weights for policy 1, policy_version 36210 (0.0010) [2023-10-12 21:29:46,330][44959] Updated weights for policy 1, policy_version 36220 (0.0008) [2023-10-12 21:29:46,443][43579] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 73924608. Throughput: 0: 1652.4, 1: 1644.7. Samples: 18500588. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-12 21:29:46,443][43579] Avg episode reward: [(0, '266.400'), (1, '261.150')] [2023-10-12 21:29:46,456][44958] Updated weights for policy 0, policy_version 36010 (0.0007) [2023-10-12 21:29:46,469][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000036224_37093376.pth... [2023-10-12 21:29:46,501][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000034688_35520512.pth [2023-10-12 21:29:46,830][44958] Updated weights for policy 0, policy_version 36020 (0.0008) [2023-10-12 21:29:47,205][44958] Updated weights for policy 0, policy_version 36030 (0.0011) [2023-10-12 21:29:47,271][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000036032_36896768.pth... [2023-10-12 21:29:47,311][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000034496_35323904.pth [2023-10-12 21:29:50,467][44959] Updated weights for policy 1, policy_version 36230 (0.0009) [2023-10-12 21:29:50,835][44959] Updated weights for policy 1, policy_version 36240 (0.0009) [2023-10-12 21:29:51,203][44959] Updated weights for policy 1, policy_version 36250 (0.0009) [2023-10-12 21:29:51,267][44958] Updated weights for policy 0, policy_version 36040 (0.0008) [2023-10-12 21:29:51,442][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 74022912. Throughput: 0: 1655.1, 1: 1644.0. Samples: 18510170. Policy #0 lag: (min: 24.0, avg: 40.6, max: 56.0) [2023-10-12 21:29:51,443][43579] Avg episode reward: [(0, '270.590'), (1, '262.710')] [2023-10-12 21:29:51,638][44958] Updated weights for policy 0, policy_version 36050 (0.0009) [2023-10-12 21:29:52,023][44958] Updated weights for policy 0, policy_version 36060 (0.0010) [2023-10-12 21:29:55,552][44959] Updated weights for policy 1, policy_version 36260 (0.0008) [2023-10-12 21:29:55,928][44959] Updated weights for policy 1, policy_version 36270 (0.0007) [2023-10-12 21:29:56,197][44958] Updated weights for policy 0, policy_version 36070 (0.0009) [2023-10-12 21:29:56,287][44959] Updated weights for policy 1, policy_version 36280 (0.0008) [2023-10-12 21:29:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 74055680. Throughput: 0: 1655.2, 1: 1638.2. Samples: 18530250. Policy #0 lag: (min: 24.0, avg: 40.6, max: 56.0) [2023-10-12 21:29:56,443][43579] Avg episode reward: [(0, '272.640'), (1, '267.990')] [2023-10-12 21:29:56,563][44958] Updated weights for policy 0, policy_version 36080 (0.0009) [2023-10-12 21:29:56,933][44958] Updated weights for policy 0, policy_version 36090 (0.0009) [2023-10-12 21:30:00,306][44959] Updated weights for policy 1, policy_version 36290 (0.0007) [2023-10-12 21:30:00,668][44959] Updated weights for policy 1, policy_version 36300 (0.0008) [2023-10-12 21:30:01,031][44959] Updated weights for policy 1, policy_version 36310 (0.0008) [2023-10-12 21:30:01,312][44958] Updated weights for policy 0, policy_version 36100 (0.0009) [2023-10-12 21:30:01,402][44959] Updated weights for policy 1, policy_version 36320 (0.0009) [2023-10-12 21:30:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 74153984. Throughput: 0: 1651.7, 1: 1639.2. Samples: 18549770. Policy #0 lag: (min: 24.0, avg: 40.6, max: 56.0) [2023-10-12 21:30:01,443][43579] Avg episode reward: [(0, '269.740'), (1, '266.360')] [2023-10-12 21:30:01,707][44958] Updated weights for policy 0, policy_version 36110 (0.0009) [2023-10-12 21:30:02,079][44958] Updated weights for policy 0, policy_version 36120 (0.0009) [2023-10-12 21:30:05,530][44959] Updated weights for policy 1, policy_version 36330 (0.0010) [2023-10-12 21:30:05,902][44959] Updated weights for policy 1, policy_version 36340 (0.0008) [2023-10-12 21:30:05,938][44958] Updated weights for policy 0, policy_version 36130 (0.0011) [2023-10-12 21:30:06,271][44959] Updated weights for policy 1, policy_version 36350 (0.0010) [2023-10-12 21:30:06,309][44958] Updated weights for policy 0, policy_version 36140 (0.0009) [2023-10-12 21:30:06,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 74219520. Throughput: 0: 1653.8, 1: 1643.7. Samples: 18559544. Policy #0 lag: (min: 24.0, avg: 40.6, max: 56.0) [2023-10-12 21:30:06,443][43579] Avg episode reward: [(0, '270.720'), (1, '267.330')] [2023-10-12 21:30:06,679][44958] Updated weights for policy 0, policy_version 36150 (0.0008) [2023-10-12 21:30:07,044][44958] Updated weights for policy 0, policy_version 36160 (0.0008) [2023-10-12 21:30:10,551][44959] Updated weights for policy 1, policy_version 36360 (0.0007) [2023-10-12 21:30:10,914][44959] Updated weights for policy 1, policy_version 36370 (0.0007) [2023-10-12 21:30:11,273][44958] Updated weights for policy 0, policy_version 36170 (0.0007) [2023-10-12 21:30:11,278][44959] Updated weights for policy 1, policy_version 36380 (0.0007) [2023-10-12 21:30:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 74285056. Throughput: 0: 1644.5, 1: 1641.3. Samples: 18579496. Policy #0 lag: (min: 24.0, avg: 40.6, max: 56.0) [2023-10-12 21:30:11,444][43579] Avg episode reward: [(0, '275.870'), (1, '272.130')] [2023-10-12 21:30:11,637][44958] Updated weights for policy 0, policy_version 36180 (0.0010) [2023-10-12 21:30:12,012][44958] Updated weights for policy 0, policy_version 36190 (0.0009) [2023-10-12 21:30:15,432][44959] Updated weights for policy 1, policy_version 36390 (0.0010) [2023-10-12 21:30:15,792][44959] Updated weights for policy 1, policy_version 36400 (0.0009) [2023-10-12 21:30:16,156][44959] Updated weights for policy 1, policy_version 36410 (0.0009) [2023-10-12 21:30:16,423][44958] Updated weights for policy 0, policy_version 36200 (0.0009) [2023-10-12 21:30:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 74350592. Throughput: 0: 1637.1, 1: 1638.7. Samples: 18598868. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-12 21:30:16,444][43579] Avg episode reward: [(0, '272.570'), (1, '268.610')] [2023-10-12 21:30:16,802][44958] Updated weights for policy 0, policy_version 36210 (0.0008) [2023-10-12 21:30:17,168][44958] Updated weights for policy 0, policy_version 36220 (0.0010) [2023-10-12 21:30:20,348][44959] Updated weights for policy 1, policy_version 36420 (0.0007) [2023-10-12 21:30:20,708][44959] Updated weights for policy 1, policy_version 36430 (0.0009) [2023-10-12 21:30:21,080][44959] Updated weights for policy 1, policy_version 36440 (0.0007) [2023-10-12 21:30:21,282][44958] Updated weights for policy 0, policy_version 36230 (0.0008) [2023-10-12 21:30:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 74416128. Throughput: 0: 1637.1, 1: 1636.6. Samples: 18608442. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-12 21:30:21,443][43579] Avg episode reward: [(0, '271.510'), (1, '270.870')] [2023-10-12 21:30:21,652][44958] Updated weights for policy 0, policy_version 36240 (0.0008) [2023-10-12 21:30:22,030][44958] Updated weights for policy 0, policy_version 36250 (0.0009) [2023-10-12 21:30:25,375][44959] Updated weights for policy 1, policy_version 36450 (0.0007) [2023-10-12 21:30:25,754][44959] Updated weights for policy 1, policy_version 36460 (0.0008) [2023-10-12 21:30:26,110][44959] Updated weights for policy 1, policy_version 36470 (0.0008) [2023-10-12 21:30:26,298][44958] Updated weights for policy 0, policy_version 36260 (0.0008) [2023-10-12 21:30:26,443][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 74448896. Throughput: 0: 1635.2, 1: 1632.4. Samples: 18628334. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-12 21:30:26,444][43579] Avg episode reward: [(0, '273.420'), (1, '269.410')] [2023-10-12 21:30:26,485][44959] Updated weights for policy 1, policy_version 36480 (0.0008) [2023-10-12 21:30:26,669][44958] Updated weights for policy 0, policy_version 36270 (0.0009) [2023-10-12 21:30:27,036][44958] Updated weights for policy 0, policy_version 36280 (0.0008) [2023-10-12 21:30:30,710][44959] Updated weights for policy 1, policy_version 36490 (0.0009) [2023-10-12 21:30:30,999][44958] Updated weights for policy 0, policy_version 36290 (0.0010) [2023-10-12 21:30:31,073][44959] Updated weights for policy 1, policy_version 36500 (0.0008) [2023-10-12 21:30:31,369][44958] Updated weights for policy 0, policy_version 36300 (0.0009) [2023-10-12 21:30:31,442][44959] Updated weights for policy 1, policy_version 36510 (0.0007) [2023-10-12 21:30:31,442][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 74514432. Throughput: 0: 1633.3, 1: 1639.8. Samples: 18647876. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-12 21:30:31,443][43579] Avg episode reward: [(0, '276.820'), (1, '274.710')] [2023-10-12 21:30:31,748][44958] Updated weights for policy 0, policy_version 36310 (0.0010) [2023-10-12 21:30:32,125][44958] Updated weights for policy 0, policy_version 36320 (0.0010) [2023-10-12 21:30:35,668][44959] Updated weights for policy 1, policy_version 36520 (0.0007) [2023-10-12 21:30:36,041][44959] Updated weights for policy 1, policy_version 36530 (0.0008) [2023-10-12 21:30:36,279][44958] Updated weights for policy 0, policy_version 36330 (0.0008) [2023-10-12 21:30:36,411][44959] Updated weights for policy 1, policy_version 36540 (0.0007) [2023-10-12 21:30:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 74579968. Throughput: 0: 1639.0, 1: 1639.8. Samples: 18657716. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-12 21:30:36,443][43579] Avg episode reward: [(0, '275.340'), (1, '279.610')] [2023-10-12 21:30:36,646][44958] Updated weights for policy 0, policy_version 36340 (0.0008) [2023-10-12 21:30:37,024][44958] Updated weights for policy 0, policy_version 36350 (0.0008) [2023-10-12 21:30:40,622][44959] Updated weights for policy 1, policy_version 36550 (0.0010) [2023-10-12 21:30:40,996][44959] Updated weights for policy 1, policy_version 36560 (0.0008) [2023-10-12 21:30:41,350][44958] Updated weights for policy 0, policy_version 36360 (0.0008) [2023-10-12 21:30:41,357][44959] Updated weights for policy 1, policy_version 36570 (0.0008) [2023-10-12 21:30:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 74645504. Throughput: 0: 1635.0, 1: 1640.9. Samples: 18677664. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:30:41,443][43579] Avg episode reward: [(0, '275.370'), (1, '279.590')] [2023-10-12 21:30:41,726][44958] Updated weights for policy 0, policy_version 36370 (0.0008) [2023-10-12 21:30:42,101][44958] Updated weights for policy 0, policy_version 36380 (0.0008) [2023-10-12 21:30:45,264][44959] Updated weights for policy 1, policy_version 36580 (0.0010) [2023-10-12 21:30:45,637][44959] Updated weights for policy 1, policy_version 36590 (0.0011) [2023-10-12 21:30:46,017][44959] Updated weights for policy 1, policy_version 36600 (0.0008) [2023-10-12 21:30:46,273][44958] Updated weights for policy 0, policy_version 36390 (0.0009) [2023-10-12 21:30:46,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 74743808. Throughput: 0: 1636.4, 1: 1639.7. Samples: 18697198. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:30:46,443][43579] Avg episode reward: [(0, '279.150'), (1, '281.820')] [2023-10-12 21:30:46,655][44958] Updated weights for policy 0, policy_version 36400 (0.0008) [2023-10-12 21:30:47,035][44958] Updated weights for policy 0, policy_version 36410 (0.0007) [2023-10-12 21:30:50,204][44959] Updated weights for policy 1, policy_version 36610 (0.0007) [2023-10-12 21:30:50,572][44959] Updated weights for policy 1, policy_version 36620 (0.0007) [2023-10-12 21:30:50,934][44959] Updated weights for policy 1, policy_version 36630 (0.0007) [2023-10-12 21:30:51,221][44958] Updated weights for policy 0, policy_version 36420 (0.0009) [2023-10-12 21:30:51,301][44959] Updated weights for policy 1, policy_version 36640 (0.0008) [2023-10-12 21:30:51,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 74809344. Throughput: 0: 1635.6, 1: 1640.6. Samples: 18706974. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:30:51,443][43579] Avg episode reward: [(0, '277.770'), (1, '281.100')] [2023-10-12 21:30:51,594][44958] Updated weights for policy 0, policy_version 36430 (0.0009) [2023-10-12 21:30:51,966][44958] Updated weights for policy 0, policy_version 36440 (0.0008) [2023-10-12 21:30:55,384][44959] Updated weights for policy 1, policy_version 36650 (0.0010) [2023-10-12 21:30:55,760][44959] Updated weights for policy 1, policy_version 36660 (0.0009) [2023-10-12 21:30:56,125][44959] Updated weights for policy 1, policy_version 36670 (0.0008) [2023-10-12 21:30:56,136][44958] Updated weights for policy 0, policy_version 36450 (0.0009) [2023-10-12 21:30:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 74874880. Throughput: 0: 1638.4, 1: 1648.4. Samples: 18727400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:30:56,444][43579] Avg episode reward: [(0, '276.820'), (1, '277.710')] [2023-10-12 21:30:56,522][44958] Updated weights for policy 0, policy_version 36460 (0.0010) [2023-10-12 21:30:56,889][44958] Updated weights for policy 0, policy_version 36470 (0.0008) [2023-10-12 21:30:57,262][44958] Updated weights for policy 0, policy_version 36480 (0.0011) [2023-10-12 21:31:00,023][44959] Updated weights for policy 1, policy_version 36680 (0.0010) [2023-10-12 21:31:00,386][44959] Updated weights for policy 1, policy_version 36690 (0.0008) [2023-10-12 21:31:00,764][44959] Updated weights for policy 1, policy_version 36700 (0.0009) [2023-10-12 21:31:01,179][44958] Updated weights for policy 0, policy_version 36490 (0.0010) [2023-10-12 21:31:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 74940416. Throughput: 0: 1638.6, 1: 1639.0. Samples: 18746362. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 21:31:01,443][43579] Avg episode reward: [(0, '273.670'), (1, '277.900')] [2023-10-12 21:31:01,558][44958] Updated weights for policy 0, policy_version 36500 (0.0008) [2023-10-12 21:31:01,932][44958] Updated weights for policy 0, policy_version 36510 (0.0009) [2023-10-12 21:31:04,984][44959] Updated weights for policy 1, policy_version 36710 (0.0007) [2023-10-12 21:31:05,356][44959] Updated weights for policy 1, policy_version 36720 (0.0009) [2023-10-12 21:31:05,730][44959] Updated weights for policy 1, policy_version 36730 (0.0008) [2023-10-12 21:31:06,182][44958] Updated weights for policy 0, policy_version 36520 (0.0009) [2023-10-12 21:31:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75005952. Throughput: 0: 1643.1, 1: 1652.0. Samples: 18756724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:31:06,443][43579] Avg episode reward: [(0, '276.300'), (1, '276.160')] [2023-10-12 21:31:06,554][44958] Updated weights for policy 0, policy_version 36530 (0.0007) [2023-10-12 21:31:06,921][44958] Updated weights for policy 0, policy_version 36540 (0.0010) [2023-10-12 21:31:09,957][44959] Updated weights for policy 1, policy_version 36740 (0.0009) [2023-10-12 21:31:10,328][44959] Updated weights for policy 1, policy_version 36750 (0.0007) [2023-10-12 21:31:10,685][44959] Updated weights for policy 1, policy_version 36760 (0.0007) [2023-10-12 21:31:11,128][44958] Updated weights for policy 0, policy_version 36550 (0.0010) [2023-10-12 21:31:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75071488. Throughput: 0: 1644.6, 1: 1655.8. Samples: 18776852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:31:11,443][43579] Avg episode reward: [(0, '279.130'), (1, '278.100')] [2023-10-12 21:31:11,504][44958] Updated weights for policy 0, policy_version 36560 (0.0009) [2023-10-12 21:31:11,881][44958] Updated weights for policy 0, policy_version 36570 (0.0010) [2023-10-12 21:31:14,832][44959] Updated weights for policy 1, policy_version 36770 (0.0009) [2023-10-12 21:31:15,207][44959] Updated weights for policy 1, policy_version 36780 (0.0008) [2023-10-12 21:31:15,568][44959] Updated weights for policy 1, policy_version 36790 (0.0008) [2023-10-12 21:31:15,949][44959] Updated weights for policy 1, policy_version 36800 (0.0007) [2023-10-12 21:31:15,977][44958] Updated weights for policy 0, policy_version 36580 (0.0007) [2023-10-12 21:31:16,355][44958] Updated weights for policy 0, policy_version 36590 (0.0009) [2023-10-12 21:31:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75137024. Throughput: 0: 1642.0, 1: 1644.7. Samples: 18795778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:31:16,444][43579] Avg episode reward: [(0, '276.600'), (1, '276.750')] [2023-10-12 21:31:16,725][44958] Updated weights for policy 0, policy_version 36600 (0.0007) [2023-10-12 21:31:20,431][44959] Updated weights for policy 1, policy_version 36810 (0.0008) [2023-10-12 21:31:20,686][44958] Updated weights for policy 0, policy_version 36610 (0.0009) [2023-10-12 21:31:20,797][44959] Updated weights for policy 1, policy_version 36820 (0.0008) [2023-10-12 21:31:21,058][44958] Updated weights for policy 0, policy_version 36620 (0.0008) [2023-10-12 21:31:21,169][44959] Updated weights for policy 1, policy_version 36830 (0.0007) [2023-10-12 21:31:21,429][44958] Updated weights for policy 0, policy_version 36630 (0.0010) [2023-10-12 21:31:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75202560. Throughput: 0: 1641.0, 1: 1652.4. Samples: 18805918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:31:21,443][43579] Avg episode reward: [(0, '278.690'), (1, '274.440')] [2023-10-12 21:31:21,800][44958] Updated weights for policy 0, policy_version 36640 (0.0008) [2023-10-12 21:31:25,231][44959] Updated weights for policy 1, policy_version 36840 (0.0009) [2023-10-12 21:31:25,598][44959] Updated weights for policy 1, policy_version 36850 (0.0008) [2023-10-12 21:31:25,951][44959] Updated weights for policy 1, policy_version 36860 (0.0008) [2023-10-12 21:31:26,090][44958] Updated weights for policy 0, policy_version 36650 (0.0008) [2023-10-12 21:31:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 75268096. Throughput: 0: 1647.9, 1: 1652.8. Samples: 18826198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:31:26,443][43579] Avg episode reward: [(0, '277.370'), (1, '271.830')] [2023-10-12 21:31:26,463][44958] Updated weights for policy 0, policy_version 36660 (0.0008) [2023-10-12 21:31:26,842][44958] Updated weights for policy 0, policy_version 36670 (0.0008) [2023-10-12 21:31:30,259][44959] Updated weights for policy 1, policy_version 36870 (0.0010) [2023-10-12 21:31:30,639][44959] Updated weights for policy 1, policy_version 36880 (0.0011) [2023-10-12 21:31:31,017][44959] Updated weights for policy 1, policy_version 36890 (0.0008) [2023-10-12 21:31:31,154][44958] Updated weights for policy 0, policy_version 36680 (0.0008) [2023-10-12 21:31:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 75333632. Throughput: 0: 1636.4, 1: 1644.6. Samples: 18844842. Policy #0 lag: (min: 1.0, avg: 5.8, max: 33.0) [2023-10-12 21:31:31,444][43579] Avg episode reward: [(0, '281.050'), (1, '267.740')] [2023-10-12 21:31:31,534][44958] Updated weights for policy 0, policy_version 36690 (0.0011) [2023-10-12 21:31:31,899][44958] Updated weights for policy 0, policy_version 36700 (0.0008) [2023-10-12 21:31:35,311][44959] Updated weights for policy 1, policy_version 36900 (0.0007) [2023-10-12 21:31:35,685][44959] Updated weights for policy 1, policy_version 36910 (0.0009) [2023-10-12 21:31:36,058][44959] Updated weights for policy 1, policy_version 36920 (0.0009) [2023-10-12 21:31:36,220][44958] Updated weights for policy 0, policy_version 36710 (0.0007) [2023-10-12 21:31:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 75399168. Throughput: 0: 1640.7, 1: 1642.7. Samples: 18854726. Policy #0 lag: (min: 1.0, avg: 5.8, max: 33.0) [2023-10-12 21:31:36,444][43579] Avg episode reward: [(0, '277.040'), (1, '272.650')] [2023-10-12 21:31:36,581][44958] Updated weights for policy 0, policy_version 36720 (0.0009) [2023-10-12 21:31:36,954][44958] Updated weights for policy 0, policy_version 36730 (0.0007) [2023-10-12 21:31:40,189][44959] Updated weights for policy 1, policy_version 36930 (0.0009) [2023-10-12 21:31:40,553][44959] Updated weights for policy 1, policy_version 36940 (0.0007) [2023-10-12 21:31:40,931][44959] Updated weights for policy 1, policy_version 36950 (0.0008) [2023-10-12 21:31:41,108][44958] Updated weights for policy 0, policy_version 36740 (0.0009) [2023-10-12 21:31:41,291][44959] Updated weights for policy 1, policy_version 36960 (0.0007) [2023-10-12 21:31:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 75464704. Throughput: 0: 1639.3, 1: 1637.8. Samples: 18874870. Policy #0 lag: (min: 1.0, avg: 5.8, max: 33.0) [2023-10-12 21:31:41,443][43579] Avg episode reward: [(0, '273.310'), (1, '272.290')] [2023-10-12 21:31:41,480][44958] Updated weights for policy 0, policy_version 36750 (0.0009) [2023-10-12 21:31:41,840][44958] Updated weights for policy 0, policy_version 36760 (0.0009) [2023-10-12 21:31:45,497][44959] Updated weights for policy 1, policy_version 36970 (0.0010) [2023-10-12 21:31:45,859][44959] Updated weights for policy 1, policy_version 36980 (0.0007) [2023-10-12 21:31:46,044][44958] Updated weights for policy 0, policy_version 36770 (0.0010) [2023-10-12 21:31:46,226][44959] Updated weights for policy 1, policy_version 36990 (0.0009) [2023-10-12 21:31:46,415][44958] Updated weights for policy 0, policy_version 36780 (0.0009) [2023-10-12 21:31:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75530240. Throughput: 0: 1638.3, 1: 1639.8. Samples: 18893876. Policy #0 lag: (min: 1.0, avg: 5.8, max: 33.0) [2023-10-12 21:31:46,443][43579] Avg episode reward: [(0, '274.570'), (1, '275.780')] [2023-10-12 21:31:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000036992_37879808.pth... [2023-10-12 21:31:46,493][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000035456_36306944.pth [2023-10-12 21:31:46,781][44958] Updated weights for policy 0, policy_version 36790 (0.0007) [2023-10-12 21:31:47,158][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000036800_37683200.pth... [2023-10-12 21:31:47,159][44958] Updated weights for policy 0, policy_version 36800 (0.0009) [2023-10-12 21:31:47,187][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000035264_36110336.pth [2023-10-12 21:31:50,356][44959] Updated weights for policy 1, policy_version 37000 (0.0009) [2023-10-12 21:31:50,724][44959] Updated weights for policy 1, policy_version 37010 (0.0010) [2023-10-12 21:31:51,085][44959] Updated weights for policy 1, policy_version 37020 (0.0008) [2023-10-12 21:31:51,431][44958] Updated weights for policy 0, policy_version 36810 (0.0007) [2023-10-12 21:31:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75595776. Throughput: 0: 1633.6, 1: 1630.2. Samples: 18903596. Policy #0 lag: (min: 1.0, avg: 5.8, max: 33.0) [2023-10-12 21:31:51,444][43579] Avg episode reward: [(0, '275.860'), (1, '276.920')] [2023-10-12 21:31:51,807][44958] Updated weights for policy 0, policy_version 36820 (0.0010) [2023-10-12 21:31:52,176][44958] Updated weights for policy 0, policy_version 36830 (0.0008) [2023-10-12 21:31:55,274][44959] Updated weights for policy 1, policy_version 37030 (0.0009) [2023-10-12 21:31:55,631][44959] Updated weights for policy 1, policy_version 37040 (0.0007) [2023-10-12 21:31:56,007][44959] Updated weights for policy 1, policy_version 37050 (0.0008) [2023-10-12 21:31:56,293][44958] Updated weights for policy 0, policy_version 36840 (0.0007) [2023-10-12 21:31:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75661312. Throughput: 0: 1637.4, 1: 1637.1. Samples: 18924204. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-12 21:31:56,444][43579] Avg episode reward: [(0, '274.580'), (1, '281.400')] [2023-10-12 21:31:56,655][44958] Updated weights for policy 0, policy_version 36850 (0.0009) [2023-10-12 21:31:57,025][44958] Updated weights for policy 0, policy_version 36860 (0.0009) [2023-10-12 21:32:00,301][44959] Updated weights for policy 1, policy_version 37060 (0.0008) [2023-10-12 21:32:00,717][44959] Updated weights for policy 1, policy_version 37070 (0.0007) [2023-10-12 21:32:01,081][44959] Updated weights for policy 1, policy_version 37080 (0.0009) [2023-10-12 21:32:01,182][44958] Updated weights for policy 0, policy_version 36870 (0.0007) [2023-10-12 21:32:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75726848. Throughput: 0: 1639.1, 1: 1637.8. Samples: 18943240. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-12 21:32:01,444][43579] Avg episode reward: [(0, '275.290'), (1, '279.310')] [2023-10-12 21:32:01,549][44958] Updated weights for policy 0, policy_version 36880 (0.0008) [2023-10-12 21:32:01,912][44958] Updated weights for policy 0, policy_version 36890 (0.0008) [2023-10-12 21:32:05,206][44959] Updated weights for policy 1, policy_version 37090 (0.0008) [2023-10-12 21:32:05,571][44959] Updated weights for policy 1, policy_version 37100 (0.0008) [2023-10-12 21:32:05,891][44958] Updated weights for policy 0, policy_version 36900 (0.0008) [2023-10-12 21:32:05,931][44959] Updated weights for policy 1, policy_version 37110 (0.0008) [2023-10-12 21:32:06,260][44958] Updated weights for policy 0, policy_version 36910 (0.0010) [2023-10-12 21:32:06,300][44959] Updated weights for policy 1, policy_version 37120 (0.0010) [2023-10-12 21:32:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75792384. Throughput: 0: 1639.4, 1: 1632.1. Samples: 18953136. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-12 21:32:06,443][43579] Avg episode reward: [(0, '276.720'), (1, '277.520')] [2023-10-12 21:32:06,648][44958] Updated weights for policy 0, policy_version 36920 (0.0011) [2023-10-12 21:32:10,367][44959] Updated weights for policy 1, policy_version 37130 (0.0007) [2023-10-12 21:32:10,729][44959] Updated weights for policy 1, policy_version 37140 (0.0009) [2023-10-12 21:32:10,864][44958] Updated weights for policy 0, policy_version 36930 (0.0007) [2023-10-12 21:32:11,102][44959] Updated weights for policy 1, policy_version 37150 (0.0009) [2023-10-12 21:32:11,239][44958] Updated weights for policy 0, policy_version 36940 (0.0008) [2023-10-12 21:32:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75857920. Throughput: 0: 1639.4, 1: 1634.1. Samples: 18973504. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-12 21:32:11,444][43579] Avg episode reward: [(0, '281.090'), (1, '272.670')] [2023-10-12 21:32:11,609][44958] Updated weights for policy 0, policy_version 36950 (0.0009) [2023-10-12 21:32:11,982][44958] Updated weights for policy 0, policy_version 36960 (0.0010) [2023-10-12 21:32:15,321][44959] Updated weights for policy 1, policy_version 37160 (0.0008) [2023-10-12 21:32:15,691][44959] Updated weights for policy 1, policy_version 37170 (0.0010) [2023-10-12 21:32:16,062][44959] Updated weights for policy 1, policy_version 37180 (0.0008) [2023-10-12 21:32:16,379][44958] Updated weights for policy 0, policy_version 36970 (0.0007) [2023-10-12 21:32:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75923456. Throughput: 0: 1647.0, 1: 1634.4. Samples: 18992504. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-10-12 21:32:16,444][43579] Avg episode reward: [(0, '275.740'), (1, '272.360')] [2023-10-12 21:32:16,746][44958] Updated weights for policy 0, policy_version 36980 (0.0007) [2023-10-12 21:32:17,119][44958] Updated weights for policy 0, policy_version 36990 (0.0009) [2023-10-12 21:32:20,373][44959] Updated weights for policy 1, policy_version 37190 (0.0009) [2023-10-12 21:32:20,737][44959] Updated weights for policy 1, policy_version 37200 (0.0008) [2023-10-12 21:32:21,103][44959] Updated weights for policy 1, policy_version 37210 (0.0008) [2023-10-12 21:32:21,223][44958] Updated weights for policy 0, policy_version 37000 (0.0009) [2023-10-12 21:32:21,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 75988992. Throughput: 0: 1646.1, 1: 1634.8. Samples: 19002368. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:32:21,443][43579] Avg episode reward: [(0, '278.270'), (1, '271.440')] [2023-10-12 21:32:21,599][44958] Updated weights for policy 0, policy_version 37010 (0.0007) [2023-10-12 21:32:21,959][44958] Updated weights for policy 0, policy_version 37020 (0.0008) [2023-10-12 21:32:25,164][44959] Updated weights for policy 1, policy_version 37220 (0.0008) [2023-10-12 21:32:25,530][44959] Updated weights for policy 1, policy_version 37230 (0.0011) [2023-10-12 21:32:25,901][44959] Updated weights for policy 1, policy_version 37240 (0.0009) [2023-10-12 21:32:26,140][44958] Updated weights for policy 0, policy_version 37030 (0.0008) [2023-10-12 21:32:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76054528. Throughput: 0: 1643.6, 1: 1636.2. Samples: 19022460. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:32:26,443][43579] Avg episode reward: [(0, '272.290'), (1, '268.130')] [2023-10-12 21:32:26,504][44958] Updated weights for policy 0, policy_version 37040 (0.0008) [2023-10-12 21:32:26,884][44958] Updated weights for policy 0, policy_version 37050 (0.0007) [2023-10-12 21:32:30,195][44959] Updated weights for policy 1, policy_version 37250 (0.0008) [2023-10-12 21:32:30,569][44959] Updated weights for policy 1, policy_version 37260 (0.0008) [2023-10-12 21:32:30,942][44959] Updated weights for policy 1, policy_version 37270 (0.0010) [2023-10-12 21:32:31,141][44958] Updated weights for policy 0, policy_version 37060 (0.0010) [2023-10-12 21:32:31,313][44959] Updated weights for policy 1, policy_version 37280 (0.0009) [2023-10-12 21:32:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 76120064. Throughput: 0: 1648.8, 1: 1635.6. Samples: 19041672. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:32:31,443][43579] Avg episode reward: [(0, '272.440'), (1, '269.070')] [2023-10-12 21:32:31,508][44958] Updated weights for policy 0, policy_version 37070 (0.0008) [2023-10-12 21:32:31,881][44958] Updated weights for policy 0, policy_version 37080 (0.0008) [2023-10-12 21:32:35,391][44959] Updated weights for policy 1, policy_version 37290 (0.0008) [2023-10-12 21:32:35,757][44959] Updated weights for policy 1, policy_version 37300 (0.0007) [2023-10-12 21:32:35,975][44958] Updated weights for policy 0, policy_version 37090 (0.0008) [2023-10-12 21:32:36,127][44959] Updated weights for policy 1, policy_version 37310 (0.0008) [2023-10-12 21:32:36,349][44958] Updated weights for policy 0, policy_version 37100 (0.0008) [2023-10-12 21:32:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76185600. Throughput: 0: 1649.4, 1: 1642.9. Samples: 19051750. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:32:36,443][43579] Avg episode reward: [(0, '272.570'), (1, '271.850')] [2023-10-12 21:32:36,721][44958] Updated weights for policy 0, policy_version 37110 (0.0008) [2023-10-12 21:32:37,088][44958] Updated weights for policy 0, policy_version 37120 (0.0009) [2023-10-12 21:32:40,117][44959] Updated weights for policy 1, policy_version 37320 (0.0008) [2023-10-12 21:32:40,488][44959] Updated weights for policy 1, policy_version 37330 (0.0010) [2023-10-12 21:32:40,856][44959] Updated weights for policy 1, policy_version 37340 (0.0008) [2023-10-12 21:32:41,182][44958] Updated weights for policy 0, policy_version 37130 (0.0007) [2023-10-12 21:32:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76251136. Throughput: 0: 1643.4, 1: 1640.4. Samples: 19071976. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:32:41,443][43579] Avg episode reward: [(0, '272.550'), (1, '277.240')] [2023-10-12 21:32:41,566][44958] Updated weights for policy 0, policy_version 37140 (0.0010) [2023-10-12 21:32:41,936][44958] Updated weights for policy 0, policy_version 37150 (0.0009) [2023-10-12 21:32:45,229][44959] Updated weights for policy 1, policy_version 37350 (0.0008) [2023-10-12 21:32:45,621][44959] Updated weights for policy 1, policy_version 37360 (0.0007) [2023-10-12 21:32:45,989][44959] Updated weights for policy 1, policy_version 37370 (0.0007) [2023-10-12 21:32:46,033][44958] Updated weights for policy 0, policy_version 37160 (0.0007) [2023-10-12 21:32:46,413][44958] Updated weights for policy 0, policy_version 37170 (0.0009) [2023-10-12 21:32:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76316672. Throughput: 0: 1641.1, 1: 1640.6. Samples: 19090918. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-12 21:32:46,443][43579] Avg episode reward: [(0, '274.730'), (1, '274.870')] [2023-10-12 21:32:46,780][44958] Updated weights for policy 0, policy_version 37180 (0.0008) [2023-10-12 21:32:50,053][44959] Updated weights for policy 1, policy_version 37380 (0.0008) [2023-10-12 21:32:50,417][44959] Updated weights for policy 1, policy_version 37390 (0.0008) [2023-10-12 21:32:50,788][44959] Updated weights for policy 1, policy_version 37400 (0.0007) [2023-10-12 21:32:50,981][44958] Updated weights for policy 0, policy_version 37190 (0.0008) [2023-10-12 21:32:51,360][44958] Updated weights for policy 0, policy_version 37200 (0.0008) [2023-10-12 21:32:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76382208. Throughput: 0: 1641.3, 1: 1646.6. Samples: 19101092. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-12 21:32:51,443][43579] Avg episode reward: [(0, '277.640'), (1, '278.080')] [2023-10-12 21:32:51,734][44958] Updated weights for policy 0, policy_version 37210 (0.0008) [2023-10-12 21:32:54,879][44959] Updated weights for policy 1, policy_version 37410 (0.0008) [2023-10-12 21:32:55,243][44959] Updated weights for policy 1, policy_version 37420 (0.0010) [2023-10-12 21:32:55,608][44959] Updated weights for policy 1, policy_version 37430 (0.0008) [2023-10-12 21:32:55,894][44958] Updated weights for policy 0, policy_version 37220 (0.0007) [2023-10-12 21:32:55,978][44959] Updated weights for policy 1, policy_version 37440 (0.0007) [2023-10-12 21:32:56,272][44958] Updated weights for policy 0, policy_version 37230 (0.0010) [2023-10-12 21:32:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76447744. Throughput: 0: 1640.1, 1: 1641.8. Samples: 19121192. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-12 21:32:56,443][43579] Avg episode reward: [(0, '274.420'), (1, '277.970')] [2023-10-12 21:32:56,642][44958] Updated weights for policy 0, policy_version 37240 (0.0008) [2023-10-12 21:33:00,089][44959] Updated weights for policy 1, policy_version 37450 (0.0008) [2023-10-12 21:33:00,450][44959] Updated weights for policy 1, policy_version 37460 (0.0009) [2023-10-12 21:33:00,786][44958] Updated weights for policy 0, policy_version 37250 (0.0007) [2023-10-12 21:33:00,817][44959] Updated weights for policy 1, policy_version 37470 (0.0007) [2023-10-12 21:33:01,188][44958] Updated weights for policy 0, policy_version 37260 (0.0009) [2023-10-12 21:33:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76513280. Throughput: 0: 1635.0, 1: 1642.8. Samples: 19140008. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-12 21:33:01,443][43579] Avg episode reward: [(0, '273.560'), (1, '282.770')] [2023-10-12 21:33:01,547][44958] Updated weights for policy 0, policy_version 37270 (0.0010) [2023-10-12 21:33:01,922][44958] Updated weights for policy 0, policy_version 37280 (0.0010) [2023-10-12 21:33:05,113][44959] Updated weights for policy 1, policy_version 37480 (0.0007) [2023-10-12 21:33:05,488][44959] Updated weights for policy 1, policy_version 37490 (0.0007) [2023-10-12 21:33:05,855][44959] Updated weights for policy 1, policy_version 37500 (0.0009) [2023-10-12 21:33:06,185][44958] Updated weights for policy 0, policy_version 37290 (0.0008) [2023-10-12 21:33:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76578816. Throughput: 0: 1636.5, 1: 1655.1. Samples: 19150490. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-12 21:33:06,443][43579] Avg episode reward: [(0, '276.350'), (1, '278.160')] [2023-10-12 21:33:06,564][44958] Updated weights for policy 0, policy_version 37300 (0.0007) [2023-10-12 21:33:06,940][44958] Updated weights for policy 0, policy_version 37310 (0.0007) [2023-10-12 21:33:09,945][44959] Updated weights for policy 1, policy_version 37510 (0.0007) [2023-10-12 21:33:10,314][44959] Updated weights for policy 1, policy_version 37520 (0.0007) [2023-10-12 21:33:10,688][44959] Updated weights for policy 1, policy_version 37530 (0.0008) [2023-10-12 21:33:11,241][44958] Updated weights for policy 0, policy_version 37320 (0.0010) [2023-10-12 21:33:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76644352. Throughput: 0: 1637.8, 1: 1641.2. Samples: 19170018. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:33:11,443][43579] Avg episode reward: [(0, '271.700'), (1, '279.290')] [2023-10-12 21:33:11,620][44958] Updated weights for policy 0, policy_version 37330 (0.0010) [2023-10-12 21:33:11,987][44958] Updated weights for policy 0, policy_version 37340 (0.0009) [2023-10-12 21:33:14,904][44959] Updated weights for policy 1, policy_version 37540 (0.0010) [2023-10-12 21:33:15,271][44959] Updated weights for policy 1, policy_version 37550 (0.0008) [2023-10-12 21:33:15,652][44959] Updated weights for policy 1, policy_version 37560 (0.0007) [2023-10-12 21:33:16,056][44958] Updated weights for policy 0, policy_version 37350 (0.0009) [2023-10-12 21:33:16,436][44958] Updated weights for policy 0, policy_version 37360 (0.0008) [2023-10-12 21:33:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76709888. Throughput: 0: 1631.1, 1: 1646.6. Samples: 19189172. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:33:16,444][43579] Avg episode reward: [(0, '272.440'), (1, '281.250')] [2023-10-12 21:33:16,811][44958] Updated weights for policy 0, policy_version 37370 (0.0010) [2023-10-12 21:33:19,853][44959] Updated weights for policy 1, policy_version 37570 (0.0008) [2023-10-12 21:33:20,222][44959] Updated weights for policy 1, policy_version 37580 (0.0008) [2023-10-12 21:33:20,594][44959] Updated weights for policy 1, policy_version 37590 (0.0007) [2023-10-12 21:33:20,966][44959] Updated weights for policy 1, policy_version 37600 (0.0010) [2023-10-12 21:33:21,156][44958] Updated weights for policy 0, policy_version 37380 (0.0009) [2023-10-12 21:33:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76775424. Throughput: 0: 1633.9, 1: 1649.6. Samples: 19199510. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:33:21,444][43579] Avg episode reward: [(0, '275.480'), (1, '283.360')] [2023-10-12 21:33:21,534][44958] Updated weights for policy 0, policy_version 37390 (0.0008) [2023-10-12 21:33:21,904][44958] Updated weights for policy 0, policy_version 37400 (0.0009) [2023-10-12 21:33:25,066][44959] Updated weights for policy 1, policy_version 37610 (0.0009) [2023-10-12 21:33:25,438][44959] Updated weights for policy 1, policy_version 37620 (0.0009) [2023-10-12 21:33:25,811][44959] Updated weights for policy 1, policy_version 37630 (0.0007) [2023-10-12 21:33:26,003][44958] Updated weights for policy 0, policy_version 37410 (0.0009) [2023-10-12 21:33:26,381][44958] Updated weights for policy 0, policy_version 37420 (0.0011) [2023-10-12 21:33:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76840960. Throughput: 0: 1631.2, 1: 1641.8. Samples: 19219264. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:33:26,444][43579] Avg episode reward: [(0, '274.790'), (1, '282.450')] [2023-10-12 21:33:26,765][44958] Updated weights for policy 0, policy_version 37430 (0.0010) [2023-10-12 21:33:27,141][44958] Updated weights for policy 0, policy_version 37440 (0.0010) [2023-10-12 21:33:29,922][44959] Updated weights for policy 1, policy_version 37640 (0.0010) [2023-10-12 21:33:30,304][44959] Updated weights for policy 1, policy_version 37650 (0.0010) [2023-10-12 21:33:30,685][44959] Updated weights for policy 1, policy_version 37660 (0.0010) [2023-10-12 21:33:31,388][44958] Updated weights for policy 0, policy_version 37450 (0.0008) [2023-10-12 21:33:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 76906496. Throughput: 0: 1633.4, 1: 1650.6. Samples: 19238698. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-12 21:33:31,444][43579] Avg episode reward: [(0, '275.500'), (1, '277.700')] [2023-10-12 21:33:31,755][44958] Updated weights for policy 0, policy_version 37460 (0.0010) [2023-10-12 21:33:32,121][44958] Updated weights for policy 0, policy_version 37470 (0.0009) [2023-10-12 21:33:34,824][44959] Updated weights for policy 1, policy_version 37670 (0.0008) [2023-10-12 21:33:35,190][44959] Updated weights for policy 1, policy_version 37680 (0.0008) [2023-10-12 21:33:35,564][44959] Updated weights for policy 1, policy_version 37690 (0.0008) [2023-10-12 21:33:36,321][44958] Updated weights for policy 0, policy_version 37480 (0.0008) [2023-10-12 21:33:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76972032. Throughput: 0: 1629.0, 1: 1655.4. Samples: 19248890. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 21:33:36,443][43579] Avg episode reward: [(0, '276.350'), (1, '281.510')] [2023-10-12 21:33:36,694][44958] Updated weights for policy 0, policy_version 37490 (0.0009) [2023-10-12 21:33:37,077][44958] Updated weights for policy 0, policy_version 37500 (0.0008) [2023-10-12 21:33:39,576][44959] Updated weights for policy 1, policy_version 37700 (0.0007) [2023-10-12 21:33:39,949][44959] Updated weights for policy 1, policy_version 37710 (0.0008) [2023-10-12 21:33:40,319][44959] Updated weights for policy 1, policy_version 37720 (0.0009) [2023-10-12 21:33:41,175][44958] Updated weights for policy 0, policy_version 37510 (0.0008) [2023-10-12 21:33:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77037568. Throughput: 0: 1631.9, 1: 1647.0. Samples: 19268742. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 21:33:41,444][43579] Avg episode reward: [(0, '277.040'), (1, '283.090')] [2023-10-12 21:33:41,549][44958] Updated weights for policy 0, policy_version 37520 (0.0008) [2023-10-12 21:33:41,920][44958] Updated weights for policy 0, policy_version 37530 (0.0009) [2023-10-12 21:33:44,482][44959] Updated weights for policy 1, policy_version 37730 (0.0009) [2023-10-12 21:33:44,848][44959] Updated weights for policy 1, policy_version 37740 (0.0007) [2023-10-12 21:33:45,225][44959] Updated weights for policy 1, policy_version 37750 (0.0008) [2023-10-12 21:33:45,596][44959] Updated weights for policy 1, policy_version 37760 (0.0007) [2023-10-12 21:33:46,139][44958] Updated weights for policy 0, policy_version 37540 (0.0010) [2023-10-12 21:33:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77103104. Throughput: 0: 1638.4, 1: 1656.1. Samples: 19288264. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 21:33:46,443][43579] Avg episode reward: [(0, '279.120'), (1, '284.290')] [2023-10-12 21:33:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000037760_38666240.pth... [2023-10-12 21:33:46,485][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000036224_37093376.pth [2023-10-12 21:33:46,519][44958] Updated weights for policy 0, policy_version 37550 (0.0011) [2023-10-12 21:33:46,882][44958] Updated weights for policy 0, policy_version 37560 (0.0011) [2023-10-12 21:33:47,176][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000037568_38469632.pth... [2023-10-12 21:33:47,212][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000036032_36896768.pth [2023-10-12 21:33:49,690][44959] Updated weights for policy 1, policy_version 37770 (0.0010) [2023-10-12 21:33:50,063][44959] Updated weights for policy 1, policy_version 37780 (0.0010) [2023-10-12 21:33:50,429][44959] Updated weights for policy 1, policy_version 37790 (0.0009) [2023-10-12 21:33:51,088][44958] Updated weights for policy 0, policy_version 37570 (0.0008) [2023-10-12 21:33:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 77168640. Throughput: 0: 1632.1, 1: 1653.4. Samples: 19298340. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 21:33:51,443][43579] Avg episode reward: [(0, '278.920'), (1, '281.480')] [2023-10-12 21:33:51,469][44958] Updated weights for policy 0, policy_version 37580 (0.0007) [2023-10-12 21:33:51,848][44958] Updated weights for policy 0, policy_version 37590 (0.0008) [2023-10-12 21:33:52,222][44958] Updated weights for policy 0, policy_version 37600 (0.0007) [2023-10-12 21:33:54,670][44959] Updated weights for policy 1, policy_version 37800 (0.0008) [2023-10-12 21:33:55,040][44959] Updated weights for policy 1, policy_version 37810 (0.0008) [2023-10-12 21:33:55,402][44959] Updated weights for policy 1, policy_version 37820 (0.0008) [2023-10-12 21:33:56,406][44958] Updated weights for policy 0, policy_version 37610 (0.0011) [2023-10-12 21:33:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77234176. Throughput: 0: 1638.0, 1: 1648.0. Samples: 19317892. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 21:33:56,443][43579] Avg episode reward: [(0, '276.850'), (1, '282.500')] [2023-10-12 21:33:56,773][44958] Updated weights for policy 0, policy_version 37620 (0.0007) [2023-10-12 21:33:57,150][44958] Updated weights for policy 0, policy_version 37630 (0.0007) [2023-10-12 21:33:59,560][44959] Updated weights for policy 1, policy_version 37830 (0.0008) [2023-10-12 21:33:59,925][44959] Updated weights for policy 1, policy_version 37840 (0.0008) [2023-10-12 21:34:00,294][44959] Updated weights for policy 1, policy_version 37850 (0.0010) [2023-10-12 21:34:01,418][44958] Updated weights for policy 0, policy_version 37640 (0.0011) [2023-10-12 21:34:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77299712. Throughput: 0: 1643.8, 1: 1654.8. Samples: 19337610. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:34:01,444][43579] Avg episode reward: [(0, '276.900'), (1, '283.590')] [2023-10-12 21:34:01,794][44958] Updated weights for policy 0, policy_version 37650 (0.0011) [2023-10-12 21:34:02,158][44958] Updated weights for policy 0, policy_version 37660 (0.0009) [2023-10-12 21:34:04,407][44959] Updated weights for policy 1, policy_version 37860 (0.0008) [2023-10-12 21:34:04,776][44959] Updated weights for policy 1, policy_version 37870 (0.0007) [2023-10-12 21:34:05,151][44959] Updated weights for policy 1, policy_version 37880 (0.0008) [2023-10-12 21:34:06,217][44958] Updated weights for policy 0, policy_version 37670 (0.0010) [2023-10-12 21:34:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77365248. Throughput: 0: 1639.1, 1: 1657.2. Samples: 19347840. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:34:06,443][43579] Avg episode reward: [(0, '278.420'), (1, '278.490')] [2023-10-12 21:34:06,586][44958] Updated weights for policy 0, policy_version 37680 (0.0009) [2023-10-12 21:34:06,963][44958] Updated weights for policy 0, policy_version 37690 (0.0007) [2023-10-12 21:34:09,351][44959] Updated weights for policy 1, policy_version 37890 (0.0009) [2023-10-12 21:34:09,724][44959] Updated weights for policy 1, policy_version 37900 (0.0008) [2023-10-12 21:34:10,090][44959] Updated weights for policy 1, policy_version 37910 (0.0008) [2023-10-12 21:34:10,454][44959] Updated weights for policy 1, policy_version 37920 (0.0007) [2023-10-12 21:34:11,130][44958] Updated weights for policy 0, policy_version 37700 (0.0009) [2023-10-12 21:34:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77430784. Throughput: 0: 1646.9, 1: 1645.9. Samples: 19367438. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:34:11,443][43579] Avg episode reward: [(0, '276.400'), (1, '273.770')] [2023-10-12 21:34:11,506][44958] Updated weights for policy 0, policy_version 37710 (0.0007) [2023-10-12 21:34:11,884][44958] Updated weights for policy 0, policy_version 37720 (0.0009) [2023-10-12 21:34:14,482][44959] Updated weights for policy 1, policy_version 37930 (0.0008) [2023-10-12 21:34:14,849][44959] Updated weights for policy 1, policy_version 37940 (0.0007) [2023-10-12 21:34:15,217][44959] Updated weights for policy 1, policy_version 37950 (0.0007) [2023-10-12 21:34:16,129][44958] Updated weights for policy 0, policy_version 37730 (0.0009) [2023-10-12 21:34:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77496320. Throughput: 0: 1645.9, 1: 1653.2. Samples: 19387156. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:34:16,443][43579] Avg episode reward: [(0, '275.690'), (1, '266.180')] [2023-10-12 21:34:16,505][44958] Updated weights for policy 0, policy_version 37740 (0.0008) [2023-10-12 21:34:16,870][44958] Updated weights for policy 0, policy_version 37750 (0.0008) [2023-10-12 21:34:17,247][44958] Updated weights for policy 0, policy_version 37760 (0.0008) [2023-10-12 21:34:19,346][44959] Updated weights for policy 1, policy_version 37960 (0.0007) [2023-10-12 21:34:19,716][44959] Updated weights for policy 1, policy_version 37970 (0.0007) [2023-10-12 21:34:20,083][44959] Updated weights for policy 1, policy_version 37980 (0.0010) [2023-10-12 21:34:21,088][44958] Updated weights for policy 0, policy_version 37770 (0.0008) [2023-10-12 21:34:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 77561856. Throughput: 0: 1644.8, 1: 1656.2. Samples: 19397438. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:34:21,444][43579] Avg episode reward: [(0, '277.960'), (1, '267.010')] [2023-10-12 21:34:21,460][44958] Updated weights for policy 0, policy_version 37780 (0.0007) [2023-10-12 21:34:21,836][44958] Updated weights for policy 0, policy_version 37790 (0.0007) [2023-10-12 21:34:24,348][44959] Updated weights for policy 1, policy_version 37990 (0.0008) [2023-10-12 21:34:24,728][44959] Updated weights for policy 1, policy_version 38000 (0.0009) [2023-10-12 21:34:25,092][44959] Updated weights for policy 1, policy_version 38010 (0.0008) [2023-10-12 21:34:25,904][44958] Updated weights for policy 0, policy_version 37800 (0.0009) [2023-10-12 21:34:26,278][44958] Updated weights for policy 0, policy_version 37810 (0.0009) [2023-10-12 21:34:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 77627392. Throughput: 0: 1648.6, 1: 1640.6. Samples: 19416758. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-12 21:34:26,443][43579] Avg episode reward: [(0, '280.930'), (1, '264.440')] [2023-10-12 21:34:26,654][44958] Updated weights for policy 0, policy_version 37820 (0.0007) [2023-10-12 21:34:29,208][44959] Updated weights for policy 1, policy_version 38020 (0.0009) [2023-10-12 21:34:29,563][44959] Updated weights for policy 1, policy_version 38030 (0.0010) [2023-10-12 21:34:29,935][44959] Updated weights for policy 1, policy_version 38040 (0.0009) [2023-10-12 21:34:31,069][44958] Updated weights for policy 0, policy_version 37830 (0.0007) [2023-10-12 21:34:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 77692928. Throughput: 0: 1641.5, 1: 1646.9. Samples: 19436244. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 21:34:31,443][43579] Avg episode reward: [(0, '281.400'), (1, '268.490')] [2023-10-12 21:34:31,458][44958] Updated weights for policy 0, policy_version 37840 (0.0009) [2023-10-12 21:34:31,820][44958] Updated weights for policy 0, policy_version 37850 (0.0008) [2023-10-12 21:34:34,150][44959] Updated weights for policy 1, policy_version 38050 (0.0008) [2023-10-12 21:34:34,517][44959] Updated weights for policy 1, policy_version 38060 (0.0007) [2023-10-12 21:34:34,886][44959] Updated weights for policy 1, policy_version 38070 (0.0007) [2023-10-12 21:34:35,246][44959] Updated weights for policy 1, policy_version 38080 (0.0007) [2023-10-12 21:34:35,900][44958] Updated weights for policy 0, policy_version 37860 (0.0008) [2023-10-12 21:34:36,274][44958] Updated weights for policy 0, policy_version 37870 (0.0008) [2023-10-12 21:34:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77758464. Throughput: 0: 1648.4, 1: 1643.7. Samples: 19446486. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 21:34:36,443][43579] Avg episode reward: [(0, '275.040'), (1, '265.630')] [2023-10-12 21:34:36,648][44958] Updated weights for policy 0, policy_version 37880 (0.0009) [2023-10-12 21:34:39,579][44959] Updated weights for policy 1, policy_version 38090 (0.0008) [2023-10-12 21:34:39,949][44959] Updated weights for policy 1, policy_version 38100 (0.0007) [2023-10-12 21:34:40,311][44959] Updated weights for policy 1, policy_version 38110 (0.0007) [2023-10-12 21:34:40,751][44958] Updated weights for policy 0, policy_version 37890 (0.0007) [2023-10-12 21:34:41,130][44958] Updated weights for policy 0, policy_version 37900 (0.0008) [2023-10-12 21:34:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77824000. Throughput: 0: 1647.3, 1: 1643.4. Samples: 19465972. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 21:34:41,443][43579] Avg episode reward: [(0, '281.040'), (1, '267.690')] [2023-10-12 21:34:41,508][44958] Updated weights for policy 0, policy_version 37910 (0.0007) [2023-10-12 21:34:41,875][44958] Updated weights for policy 0, policy_version 37920 (0.0008) [2023-10-12 21:34:44,239][44959] Updated weights for policy 1, policy_version 38120 (0.0010) [2023-10-12 21:34:44,612][44959] Updated weights for policy 1, policy_version 38130 (0.0008) [2023-10-12 21:34:44,979][44959] Updated weights for policy 1, policy_version 38140 (0.0007) [2023-10-12 21:34:46,030][44958] Updated weights for policy 0, policy_version 37930 (0.0008) [2023-10-12 21:34:46,409][44958] Updated weights for policy 0, policy_version 37940 (0.0008) [2023-10-12 21:34:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 77889536. Throughput: 0: 1634.8, 1: 1649.1. Samples: 19485382. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 21:34:46,444][43579] Avg episode reward: [(0, '279.380'), (1, '273.840')] [2023-10-12 21:34:46,785][44958] Updated weights for policy 0, policy_version 37950 (0.0010) [2023-10-12 21:34:49,109][44959] Updated weights for policy 1, policy_version 38150 (0.0010) [2023-10-12 21:34:49,481][44959] Updated weights for policy 1, policy_version 38160 (0.0011) [2023-10-12 21:34:49,845][44959] Updated weights for policy 1, policy_version 38170 (0.0007) [2023-10-12 21:34:51,075][44958] Updated weights for policy 0, policy_version 37960 (0.0011) [2023-10-12 21:34:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 77955072. Throughput: 0: 1642.4, 1: 1640.0. Samples: 19495550. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 21:34:51,443][43579] Avg episode reward: [(0, '277.520'), (1, '277.630')] [2023-10-12 21:34:51,463][44958] Updated weights for policy 0, policy_version 37970 (0.0010) [2023-10-12 21:34:51,832][44958] Updated weights for policy 0, policy_version 37980 (0.0012) [2023-10-12 21:34:54,126][44959] Updated weights for policy 1, policy_version 38180 (0.0010) [2023-10-12 21:34:54,490][44959] Updated weights for policy 1, policy_version 38190 (0.0009) [2023-10-12 21:34:54,856][44959] Updated weights for policy 1, policy_version 38200 (0.0010) [2023-10-12 21:34:55,856][44958] Updated weights for policy 0, policy_version 37990 (0.0010) [2023-10-12 21:34:56,235][44958] Updated weights for policy 0, policy_version 38000 (0.0009) [2023-10-12 21:34:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78020608. Throughput: 0: 1643.7, 1: 1642.0. Samples: 19515298. Policy #0 lag: (min: 18.0, avg: 18.0, max: 21.0) [2023-10-12 21:34:56,443][43579] Avg episode reward: [(0, '278.080'), (1, '279.000')] [2023-10-12 21:34:56,601][44958] Updated weights for policy 0, policy_version 38010 (0.0007) [2023-10-12 21:34:58,999][44959] Updated weights for policy 1, policy_version 38210 (0.0009) [2023-10-12 21:34:59,362][44959] Updated weights for policy 1, policy_version 38220 (0.0010) [2023-10-12 21:34:59,730][44959] Updated weights for policy 1, policy_version 38230 (0.0007) [2023-10-12 21:35:00,099][44959] Updated weights for policy 1, policy_version 38240 (0.0007) [2023-10-12 21:35:00,856][44958] Updated weights for policy 0, policy_version 38020 (0.0008) [2023-10-12 21:35:01,230][44958] Updated weights for policy 0, policy_version 38030 (0.0008) [2023-10-12 21:35:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78086144. Throughput: 0: 1639.9, 1: 1646.5. Samples: 19535048. Policy #0 lag: (min: 18.0, avg: 18.0, max: 21.0) [2023-10-12 21:35:01,444][43579] Avg episode reward: [(0, '278.000'), (1, '274.380')] [2023-10-12 21:35:01,604][44958] Updated weights for policy 0, policy_version 38040 (0.0007) [2023-10-12 21:35:04,561][44959] Updated weights for policy 1, policy_version 38250 (0.0010) [2023-10-12 21:35:04,946][44959] Updated weights for policy 1, policy_version 38260 (0.0009) [2023-10-12 21:35:05,311][44959] Updated weights for policy 1, policy_version 38270 (0.0009) [2023-10-12 21:35:05,839][44958] Updated weights for policy 0, policy_version 38050 (0.0007) [2023-10-12 21:35:06,217][44958] Updated weights for policy 0, policy_version 38060 (0.0007) [2023-10-12 21:35:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78151680. Throughput: 0: 1644.0, 1: 1642.5. Samples: 19545332. Policy #0 lag: (min: 18.0, avg: 18.0, max: 21.0) [2023-10-12 21:35:06,443][43579] Avg episode reward: [(0, '282.150'), (1, '277.640')] [2023-10-12 21:35:06,585][44958] Updated weights for policy 0, policy_version 38070 (0.0007) [2023-10-12 21:35:06,964][44958] Updated weights for policy 0, policy_version 38080 (0.0008) [2023-10-12 21:35:09,128][44959] Updated weights for policy 1, policy_version 38280 (0.0010) [2023-10-12 21:35:09,494][44959] Updated weights for policy 1, policy_version 38290 (0.0011) [2023-10-12 21:35:09,862][44959] Updated weights for policy 1, policy_version 38300 (0.0011) [2023-10-12 21:35:10,916][44958] Updated weights for policy 0, policy_version 38090 (0.0009) [2023-10-12 21:35:11,290][44958] Updated weights for policy 0, policy_version 38100 (0.0011) [2023-10-12 21:35:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78217216. Throughput: 0: 1639.7, 1: 1641.6. Samples: 19564418. Policy #0 lag: (min: 18.0, avg: 18.0, max: 21.0) [2023-10-12 21:35:11,443][43579] Avg episode reward: [(0, '279.170'), (1, '280.860')] [2023-10-12 21:35:11,671][44958] Updated weights for policy 0, policy_version 38110 (0.0007) [2023-10-12 21:35:14,068][44959] Updated weights for policy 1, policy_version 38310 (0.0009) [2023-10-12 21:35:14,430][44959] Updated weights for policy 1, policy_version 38320 (0.0007) [2023-10-12 21:35:14,806][44959] Updated weights for policy 1, policy_version 38330 (0.0007) [2023-10-12 21:35:15,889][44958] Updated weights for policy 0, policy_version 38120 (0.0008) [2023-10-12 21:35:16,269][44958] Updated weights for policy 0, policy_version 38130 (0.0010) [2023-10-12 21:35:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78282752. Throughput: 0: 1634.6, 1: 1644.1. Samples: 19583788. Policy #0 lag: (min: 18.0, avg: 18.0, max: 21.0) [2023-10-12 21:35:16,443][43579] Avg episode reward: [(0, '280.040'), (1, '280.670')] [2023-10-12 21:35:16,640][44958] Updated weights for policy 0, policy_version 38140 (0.0007) [2023-10-12 21:35:19,147][44959] Updated weights for policy 1, policy_version 38340 (0.0010) [2023-10-12 21:35:19,519][44959] Updated weights for policy 1, policy_version 38350 (0.0008) [2023-10-12 21:35:19,877][44959] Updated weights for policy 1, policy_version 38360 (0.0010) [2023-10-12 21:35:20,822][44958] Updated weights for policy 0, policy_version 38150 (0.0007) [2023-10-12 21:35:21,193][44958] Updated weights for policy 0, policy_version 38160 (0.0008) [2023-10-12 21:35:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 78348288. Throughput: 0: 1637.0, 1: 1648.0. Samples: 19594310. Policy #0 lag: (min: 18.0, avg: 18.0, max: 21.0) [2023-10-12 21:35:21,443][43579] Avg episode reward: [(0, '279.230'), (1, '280.140')] [2023-10-12 21:35:21,558][44958] Updated weights for policy 0, policy_version 38170 (0.0009) [2023-10-12 21:35:23,980][44959] Updated weights for policy 1, policy_version 38370 (0.0010) [2023-10-12 21:35:24,353][44959] Updated weights for policy 1, policy_version 38380 (0.0011) [2023-10-12 21:35:24,718][44959] Updated weights for policy 1, policy_version 38390 (0.0009) [2023-10-12 21:35:25,077][44959] Updated weights for policy 1, policy_version 38400 (0.0008) [2023-10-12 21:35:25,748][44958] Updated weights for policy 0, policy_version 38180 (0.0008) [2023-10-12 21:35:26,119][44958] Updated weights for policy 0, policy_version 38190 (0.0007) [2023-10-12 21:35:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 78413824. Throughput: 0: 1646.7, 1: 1640.7. Samples: 19613902. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:35:26,443][43579] Avg episode reward: [(0, '278.700'), (1, '286.060')] [2023-10-12 21:35:26,490][44958] Updated weights for policy 0, policy_version 38200 (0.0010) [2023-10-12 21:35:29,208][44959] Updated weights for policy 1, policy_version 38410 (0.0008) [2023-10-12 21:35:29,576][44959] Updated weights for policy 1, policy_version 38420 (0.0008) [2023-10-12 21:35:29,940][44959] Updated weights for policy 1, policy_version 38430 (0.0007) [2023-10-12 21:35:30,543][44958] Updated weights for policy 0, policy_version 38210 (0.0008) [2023-10-12 21:35:30,919][44958] Updated weights for policy 0, policy_version 38220 (0.0009) [2023-10-12 21:35:31,296][44958] Updated weights for policy 0, policy_version 38230 (0.0009) [2023-10-12 21:35:31,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 78479360. Throughput: 0: 1644.5, 1: 1640.6. Samples: 19633212. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:35:31,444][43579] Avg episode reward: [(0, '276.670'), (1, '289.220')] [2023-10-12 21:35:31,453][44583] Saving new best policy, reward=289.220! [2023-10-12 21:35:31,663][44958] Updated weights for policy 0, policy_version 38240 (0.0011) [2023-10-12 21:35:34,177][44959] Updated weights for policy 1, policy_version 38440 (0.0008) [2023-10-12 21:35:34,544][44959] Updated weights for policy 1, policy_version 38450 (0.0008) [2023-10-12 21:35:34,917][44959] Updated weights for policy 1, policy_version 38460 (0.0008) [2023-10-12 21:35:35,947][44958] Updated weights for policy 0, policy_version 38250 (0.0008) [2023-10-12 21:35:36,330][44958] Updated weights for policy 0, policy_version 38260 (0.0008) [2023-10-12 21:35:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 78544896. Throughput: 0: 1645.4, 1: 1647.5. Samples: 19643730. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:35:36,444][43579] Avg episode reward: [(0, '277.220'), (1, '288.160')] [2023-10-12 21:35:36,699][44958] Updated weights for policy 0, policy_version 38270 (0.0010) [2023-10-12 21:35:39,050][44959] Updated weights for policy 1, policy_version 38470 (0.0012) [2023-10-12 21:35:39,411][44959] Updated weights for policy 1, policy_version 38480 (0.0008) [2023-10-12 21:35:39,773][44959] Updated weights for policy 1, policy_version 38490 (0.0008) [2023-10-12 21:35:40,873][44958] Updated weights for policy 0, policy_version 38280 (0.0010) [2023-10-12 21:35:41,245][44958] Updated weights for policy 0, policy_version 38290 (0.0011) [2023-10-12 21:35:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78610432. Throughput: 0: 1643.1, 1: 1639.9. Samples: 19663032. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:35:41,444][43579] Avg episode reward: [(0, '278.420'), (1, '287.730')] [2023-10-12 21:35:41,627][44958] Updated weights for policy 0, policy_version 38300 (0.0010) [2023-10-12 21:35:44,029][44959] Updated weights for policy 1, policy_version 38500 (0.0008) [2023-10-12 21:35:44,388][44959] Updated weights for policy 1, policy_version 38510 (0.0007) [2023-10-12 21:35:44,761][44959] Updated weights for policy 1, policy_version 38520 (0.0010) [2023-10-12 21:35:45,809][44958] Updated weights for policy 0, policy_version 38310 (0.0010) [2023-10-12 21:35:46,176][44958] Updated weights for policy 0, policy_version 38320 (0.0009) [2023-10-12 21:35:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78675968. Throughput: 0: 1639.2, 1: 1643.7. Samples: 19682776. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:35:46,443][43579] Avg episode reward: [(0, '277.060'), (1, '288.190')] [2023-10-12 21:35:46,450][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000038528_39452672.pth... [2023-10-12 21:35:46,480][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000036992_37879808.pth [2023-10-12 21:35:46,484][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000038528_39452672.pth [2023-10-12 21:35:46,550][44958] Updated weights for policy 0, policy_version 38330 (0.0010) [2023-10-12 21:35:46,774][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000038336_39256064.pth... [2023-10-12 21:35:46,815][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000036800_37683200.pth [2023-10-12 21:35:46,820][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000038336_39256064.pth [2023-10-12 21:35:48,934][44959] Updated weights for policy 1, policy_version 38530 (0.0009) [2023-10-12 21:35:49,310][44959] Updated weights for policy 1, policy_version 38540 (0.0008) [2023-10-12 21:35:49,670][44959] Updated weights for policy 1, policy_version 38550 (0.0008) [2023-10-12 21:35:50,039][44959] Updated weights for policy 1, policy_version 38560 (0.0009) [2023-10-12 21:35:50,621][44958] Updated weights for policy 0, policy_version 38340 (0.0009) [2023-10-12 21:35:50,997][44958] Updated weights for policy 0, policy_version 38350 (0.0010) [2023-10-12 21:35:51,374][44958] Updated weights for policy 0, policy_version 38360 (0.0011) [2023-10-12 21:35:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78741504. Throughput: 0: 1644.2, 1: 1641.5. Samples: 19693188. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-12 21:35:51,444][43579] Avg episode reward: [(0, '276.750'), (1, '286.280')] [2023-10-12 21:35:54,075][44959] Updated weights for policy 1, policy_version 38570 (0.0009) [2023-10-12 21:35:54,438][44959] Updated weights for policy 1, policy_version 38580 (0.0008) [2023-10-12 21:35:54,803][44959] Updated weights for policy 1, policy_version 38590 (0.0010) [2023-10-12 21:35:55,574][44958] Updated weights for policy 0, policy_version 38370 (0.0011) [2023-10-12 21:35:55,941][44958] Updated weights for policy 0, policy_version 38380 (0.0009) [2023-10-12 21:35:56,306][44958] Updated weights for policy 0, policy_version 38390 (0.0010) [2023-10-12 21:35:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78807040. Throughput: 0: 1641.3, 1: 1650.2. Samples: 19712534. Policy #0 lag: (min: 16.0, avg: 37.5, max: 48.0) [2023-10-12 21:35:56,443][43579] Avg episode reward: [(0, '275.950'), (1, '286.160')] [2023-10-12 21:35:56,681][44958] Updated weights for policy 0, policy_version 38400 (0.0010) [2023-10-12 21:35:59,067][44959] Updated weights for policy 1, policy_version 38600 (0.0008) [2023-10-12 21:35:59,429][44959] Updated weights for policy 1, policy_version 38610 (0.0007) [2023-10-12 21:35:59,809][44959] Updated weights for policy 1, policy_version 38620 (0.0008) [2023-10-12 21:36:01,164][44958] Updated weights for policy 0, policy_version 38410 (0.0009) [2023-10-12 21:36:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78872576. Throughput: 0: 1638.4, 1: 1655.5. Samples: 19732012. Policy #0 lag: (min: 16.0, avg: 37.5, max: 48.0) [2023-10-12 21:36:01,443][43579] Avg episode reward: [(0, '280.100'), (1, '284.840')] [2023-10-12 21:36:01,542][44958] Updated weights for policy 0, policy_version 38420 (0.0008) [2023-10-12 21:36:01,909][44958] Updated weights for policy 0, policy_version 38430 (0.0008) [2023-10-12 21:36:03,935][44959] Updated weights for policy 1, policy_version 38630 (0.0011) [2023-10-12 21:36:04,298][44959] Updated weights for policy 1, policy_version 38640 (0.0009) [2023-10-12 21:36:04,657][44959] Updated weights for policy 1, policy_version 38650 (0.0007) [2023-10-12 21:36:05,876][44958] Updated weights for policy 0, policy_version 38440 (0.0008) [2023-10-12 21:36:06,238][44958] Updated weights for policy 0, policy_version 38450 (0.0008) [2023-10-12 21:36:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 78938112. Throughput: 0: 1640.0, 1: 1645.0. Samples: 19742138. Policy #0 lag: (min: 16.0, avg: 37.5, max: 48.0) [2023-10-12 21:36:06,443][43579] Avg episode reward: [(0, '277.050'), (1, '282.890')] [2023-10-12 21:36:06,621][44958] Updated weights for policy 0, policy_version 38460 (0.0007) [2023-10-12 21:36:08,817][44959] Updated weights for policy 1, policy_version 38660 (0.0009) [2023-10-12 21:36:09,186][44959] Updated weights for policy 1, policy_version 38670 (0.0009) [2023-10-12 21:36:09,547][44959] Updated weights for policy 1, policy_version 38680 (0.0007) [2023-10-12 21:36:11,031][44958] Updated weights for policy 0, policy_version 38470 (0.0009) [2023-10-12 21:36:11,392][44958] Updated weights for policy 0, policy_version 38480 (0.0008) [2023-10-12 21:36:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79003648. Throughput: 0: 1629.6, 1: 1651.8. Samples: 19761566. Policy #0 lag: (min: 16.0, avg: 37.5, max: 48.0) [2023-10-12 21:36:11,443][43579] Avg episode reward: [(0, '272.060'), (1, '278.850')] [2023-10-12 21:36:11,770][44958] Updated weights for policy 0, policy_version 38490 (0.0008) [2023-10-12 21:36:13,568][44959] Updated weights for policy 1, policy_version 38690 (0.0007) [2023-10-12 21:36:13,934][44959] Updated weights for policy 1, policy_version 38700 (0.0009) [2023-10-12 21:36:14,295][44959] Updated weights for policy 1, policy_version 38710 (0.0011) [2023-10-12 21:36:14,663][44959] Updated weights for policy 1, policy_version 38720 (0.0009) [2023-10-12 21:36:16,003][44958] Updated weights for policy 0, policy_version 38500 (0.0008) [2023-10-12 21:36:16,365][44958] Updated weights for policy 0, policy_version 38510 (0.0010) [2023-10-12 21:36:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79069184. Throughput: 0: 1636.2, 1: 1654.4. Samples: 19781290. Policy #0 lag: (min: 16.0, avg: 37.5, max: 48.0) [2023-10-12 21:36:16,443][43579] Avg episode reward: [(0, '273.120'), (1, '277.800')] [2023-10-12 21:36:16,740][44958] Updated weights for policy 0, policy_version 38520 (0.0009) [2023-10-12 21:36:18,821][44959] Updated weights for policy 1, policy_version 38730 (0.0008) [2023-10-12 21:36:19,181][44959] Updated weights for policy 1, policy_version 38740 (0.0008) [2023-10-12 21:36:19,551][44959] Updated weights for policy 1, policy_version 38750 (0.0009) [2023-10-12 21:36:20,867][44958] Updated weights for policy 0, policy_version 38530 (0.0010) [2023-10-12 21:36:21,236][44958] Updated weights for policy 0, policy_version 38540 (0.0007) [2023-10-12 21:36:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79134720. Throughput: 0: 1633.2, 1: 1646.5. Samples: 19791318. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 21:36:21,443][43579] Avg episode reward: [(0, '273.660'), (1, '279.690')] [2023-10-12 21:36:21,608][44958] Updated weights for policy 0, policy_version 38550 (0.0007) [2023-10-12 21:36:21,978][44958] Updated weights for policy 0, policy_version 38560 (0.0007) [2023-10-12 21:36:23,902][44959] Updated weights for policy 1, policy_version 38760 (0.0007) [2023-10-12 21:36:24,278][44959] Updated weights for policy 1, policy_version 38770 (0.0009) [2023-10-12 21:36:24,645][44959] Updated weights for policy 1, policy_version 38780 (0.0007) [2023-10-12 21:36:26,163][44958] Updated weights for policy 0, policy_version 38570 (0.0009) [2023-10-12 21:36:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79200256. Throughput: 0: 1630.5, 1: 1650.7. Samples: 19810688. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 21:36:26,444][43579] Avg episode reward: [(0, '270.260'), (1, '278.660')] [2023-10-12 21:36:26,541][44958] Updated weights for policy 0, policy_version 38580 (0.0008) [2023-10-12 21:36:26,914][44958] Updated weights for policy 0, policy_version 38590 (0.0009) [2023-10-12 21:36:28,613][44959] Updated weights for policy 1, policy_version 38790 (0.0009) [2023-10-12 21:36:28,982][44959] Updated weights for policy 1, policy_version 38800 (0.0009) [2023-10-12 21:36:29,348][44959] Updated weights for policy 1, policy_version 38810 (0.0007) [2023-10-12 21:36:31,104][44958] Updated weights for policy 0, policy_version 38600 (0.0008) [2023-10-12 21:36:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 79265792. Throughput: 0: 1630.3, 1: 1649.4. Samples: 19830362. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 21:36:31,443][43579] Avg episode reward: [(0, '270.110'), (1, '280.140')] [2023-10-12 21:36:31,473][44958] Updated weights for policy 0, policy_version 38610 (0.0008) [2023-10-12 21:36:31,854][44958] Updated weights for policy 0, policy_version 38620 (0.0008) [2023-10-12 21:36:33,616][44959] Updated weights for policy 1, policy_version 38820 (0.0008) [2023-10-12 21:36:33,979][44959] Updated weights for policy 1, policy_version 38830 (0.0008) [2023-10-12 21:36:34,345][44959] Updated weights for policy 1, policy_version 38840 (0.0009) [2023-10-12 21:36:35,904][44958] Updated weights for policy 0, policy_version 38630 (0.0008) [2023-10-12 21:36:36,271][44958] Updated weights for policy 0, policy_version 38640 (0.0009) [2023-10-12 21:36:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79331328. Throughput: 0: 1632.6, 1: 1645.3. Samples: 19840692. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 21:36:36,443][43579] Avg episode reward: [(0, '273.830'), (1, '276.650')] [2023-10-12 21:36:36,637][44958] Updated weights for policy 0, policy_version 38650 (0.0009) [2023-10-12 21:36:38,580][44959] Updated weights for policy 1, policy_version 38850 (0.0009) [2023-10-12 21:36:38,984][44959] Updated weights for policy 1, policy_version 38860 (0.0011) [2023-10-12 21:36:39,360][44959] Updated weights for policy 1, policy_version 38870 (0.0009) [2023-10-12 21:36:39,722][44959] Updated weights for policy 1, policy_version 38880 (0.0008) [2023-10-12 21:36:40,853][44958] Updated weights for policy 0, policy_version 38660 (0.0007) [2023-10-12 21:36:41,236][44958] Updated weights for policy 0, policy_version 38670 (0.0009) [2023-10-12 21:36:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79396864. Throughput: 0: 1633.1, 1: 1648.7. Samples: 19860216. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 21:36:41,444][43579] Avg episode reward: [(0, '273.350'), (1, '272.990')] [2023-10-12 21:36:41,600][44958] Updated weights for policy 0, policy_version 38680 (0.0008) [2023-10-12 21:36:43,781][44959] Updated weights for policy 1, policy_version 38890 (0.0009) [2023-10-12 21:36:44,152][44959] Updated weights for policy 1, policy_version 38900 (0.0008) [2023-10-12 21:36:44,523][44959] Updated weights for policy 1, policy_version 38910 (0.0009) [2023-10-12 21:36:45,823][44958] Updated weights for policy 0, policy_version 38690 (0.0008) [2023-10-12 21:36:46,222][44958] Updated weights for policy 0, policy_version 38700 (0.0008) [2023-10-12 21:36:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79462400. Throughput: 0: 1640.1, 1: 1655.1. Samples: 19880296. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 21:36:46,443][43579] Avg episode reward: [(0, '274.360'), (1, '268.380')] [2023-10-12 21:36:46,602][44958] Updated weights for policy 0, policy_version 38710 (0.0009) [2023-10-12 21:36:46,966][44958] Updated weights for policy 0, policy_version 38720 (0.0007) [2023-10-12 21:36:48,631][44959] Updated weights for policy 1, policy_version 38920 (0.0009) [2023-10-12 21:36:49,006][44959] Updated weights for policy 1, policy_version 38930 (0.0007) [2023-10-12 21:36:49,369][44959] Updated weights for policy 1, policy_version 38940 (0.0009) [2023-10-12 21:36:51,062][44958] Updated weights for policy 0, policy_version 38730 (0.0010) [2023-10-12 21:36:51,441][44958] Updated weights for policy 0, policy_version 38740 (0.0007) [2023-10-12 21:36:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79527936. Throughput: 0: 1640.1, 1: 1648.8. Samples: 19890140. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-12 21:36:51,444][43579] Avg episode reward: [(0, '275.650'), (1, '264.460')] [2023-10-12 21:36:51,798][44958] Updated weights for policy 0, policy_version 38750 (0.0008) [2023-10-12 21:36:53,422][44959] Updated weights for policy 1, policy_version 38950 (0.0011) [2023-10-12 21:36:53,795][44959] Updated weights for policy 1, policy_version 38960 (0.0009) [2023-10-12 21:36:54,165][44959] Updated weights for policy 1, policy_version 38970 (0.0007) [2023-10-12 21:36:55,945][44958] Updated weights for policy 0, policy_version 38760 (0.0010) [2023-10-12 21:36:56,317][44958] Updated weights for policy 0, policy_version 38770 (0.0009) [2023-10-12 21:36:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79593472. Throughput: 0: 1641.6, 1: 1655.4. Samples: 19909928. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-12 21:36:56,443][43579] Avg episode reward: [(0, '277.380'), (1, '263.150')] [2023-10-12 21:36:56,685][44958] Updated weights for policy 0, policy_version 38780 (0.0009) [2023-10-12 21:36:58,326][44959] Updated weights for policy 1, policy_version 38980 (0.0009) [2023-10-12 21:36:58,692][44959] Updated weights for policy 1, policy_version 38990 (0.0010) [2023-10-12 21:36:59,067][44959] Updated weights for policy 1, policy_version 39000 (0.0009) [2023-10-12 21:37:00,714][44958] Updated weights for policy 0, policy_version 38790 (0.0010) [2023-10-12 21:37:01,088][44958] Updated weights for policy 0, policy_version 38800 (0.0008) [2023-10-12 21:37:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79659008. Throughput: 0: 1636.4, 1: 1662.3. Samples: 19929730. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-12 21:37:01,444][43579] Avg episode reward: [(0, '274.990'), (1, '267.030')] [2023-10-12 21:37:01,465][44958] Updated weights for policy 0, policy_version 38810 (0.0009) [2023-10-12 21:37:03,280][44959] Updated weights for policy 1, policy_version 39010 (0.0007) [2023-10-12 21:37:03,648][44959] Updated weights for policy 1, policy_version 39020 (0.0008) [2023-10-12 21:37:04,013][44959] Updated weights for policy 1, policy_version 39030 (0.0009) [2023-10-12 21:37:04,386][44959] Updated weights for policy 1, policy_version 39040 (0.0008) [2023-10-12 21:37:05,787][44958] Updated weights for policy 0, policy_version 38820 (0.0008) [2023-10-12 21:37:06,162][44958] Updated weights for policy 0, policy_version 38830 (0.0009) [2023-10-12 21:37:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79724544. Throughput: 0: 1645.2, 1: 1649.2. Samples: 19939570. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-12 21:37:06,443][43579] Avg episode reward: [(0, '276.270'), (1, '268.110')] [2023-10-12 21:37:06,534][44958] Updated weights for policy 0, policy_version 38840 (0.0008) [2023-10-12 21:37:08,294][44959] Updated weights for policy 1, policy_version 39050 (0.0009) [2023-10-12 21:37:08,665][44959] Updated weights for policy 1, policy_version 39060 (0.0009) [2023-10-12 21:37:09,042][44959] Updated weights for policy 1, policy_version 39070 (0.0009) [2023-10-12 21:37:10,647][44958] Updated weights for policy 0, policy_version 38850 (0.0007) [2023-10-12 21:37:11,023][44958] Updated weights for policy 0, policy_version 38860 (0.0008) [2023-10-12 21:37:11,393][44958] Updated weights for policy 0, policy_version 38870 (0.0008) [2023-10-12 21:37:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79790080. Throughput: 0: 1647.3, 1: 1666.6. Samples: 19959814. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-12 21:37:11,443][43579] Avg episode reward: [(0, '275.230'), (1, '272.970')] [2023-10-12 21:37:11,763][44958] Updated weights for policy 0, policy_version 38880 (0.0010) [2023-10-12 21:37:13,222][44959] Updated weights for policy 1, policy_version 39080 (0.0008) [2023-10-12 21:37:13,589][44959] Updated weights for policy 1, policy_version 39090 (0.0008) [2023-10-12 21:37:13,964][44959] Updated weights for policy 1, policy_version 39100 (0.0008) [2023-10-12 21:37:15,846][44958] Updated weights for policy 0, policy_version 38890 (0.0010) [2023-10-12 21:37:16,218][44958] Updated weights for policy 0, policy_version 38900 (0.0009) [2023-10-12 21:37:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79855616. Throughput: 0: 1644.4, 1: 1668.6. Samples: 19979446. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-10-12 21:37:16,443][43579] Avg episode reward: [(0, '267.330'), (1, '273.370')] [2023-10-12 21:37:16,599][44958] Updated weights for policy 0, policy_version 38910 (0.0009) [2023-10-12 21:37:18,244][44959] Updated weights for policy 1, policy_version 39110 (0.0009) [2023-10-12 21:37:18,610][44959] Updated weights for policy 1, policy_version 39120 (0.0011) [2023-10-12 21:37:18,984][44959] Updated weights for policy 1, policy_version 39130 (0.0008) [2023-10-12 21:37:20,818][44958] Updated weights for policy 0, policy_version 38920 (0.0008) [2023-10-12 21:37:21,187][44958] Updated weights for policy 0, policy_version 38930 (0.0010) [2023-10-12 21:37:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79921152. Throughput: 0: 1646.8, 1: 1653.0. Samples: 19989184. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-12 21:37:21,443][43579] Avg episode reward: [(0, '267.060'), (1, '275.380')] [2023-10-12 21:37:21,564][44958] Updated weights for policy 0, policy_version 38940 (0.0011) [2023-10-12 21:37:23,065][44959] Updated weights for policy 1, policy_version 39140 (0.0007) [2023-10-12 21:37:23,431][44959] Updated weights for policy 1, policy_version 39150 (0.0009) [2023-10-12 21:37:23,808][44959] Updated weights for policy 1, policy_version 39160 (0.0009) [2023-10-12 21:37:25,628][44958] Updated weights for policy 0, policy_version 38950 (0.0008) [2023-10-12 21:37:25,998][44958] Updated weights for policy 0, policy_version 38960 (0.0009) [2023-10-12 21:37:26,363][44958] Updated weights for policy 0, policy_version 38970 (0.0011) [2023-10-12 21:37:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 79986688. Throughput: 0: 1643.6, 1: 1664.1. Samples: 20009064. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-12 21:37:26,443][43579] Avg episode reward: [(0, '267.060'), (1, '274.460')] [2023-10-12 21:37:28,046][44959] Updated weights for policy 1, policy_version 39170 (0.0011) [2023-10-12 21:37:28,471][44959] Updated weights for policy 1, policy_version 39180 (0.0010) [2023-10-12 21:37:28,830][44959] Updated weights for policy 1, policy_version 39190 (0.0008) [2023-10-12 21:37:29,195][44959] Updated weights for policy 1, policy_version 39200 (0.0007) [2023-10-12 21:37:30,618][44958] Updated weights for policy 0, policy_version 38980 (0.0009) [2023-10-12 21:37:31,020][44958] Updated weights for policy 0, policy_version 38990 (0.0008) [2023-10-12 21:37:31,391][44958] Updated weights for policy 0, policy_version 39000 (0.0009) [2023-10-12 21:37:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80052224. Throughput: 0: 1637.9, 1: 1650.5. Samples: 20028276. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-12 21:37:31,443][43579] Avg episode reward: [(0, '265.540'), (1, '275.490')] [2023-10-12 21:37:33,455][44959] Updated weights for policy 1, policy_version 39210 (0.0007) [2023-10-12 21:37:33,817][44959] Updated weights for policy 1, policy_version 39220 (0.0010) [2023-10-12 21:37:34,197][44959] Updated weights for policy 1, policy_version 39230 (0.0010) [2023-10-12 21:37:35,491][44958] Updated weights for policy 0, policy_version 39010 (0.0010) [2023-10-12 21:37:35,850][44958] Updated weights for policy 0, policy_version 39020 (0.0007) [2023-10-12 21:37:36,233][44958] Updated weights for policy 0, policy_version 39030 (0.0009) [2023-10-12 21:37:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80117760. Throughput: 0: 1643.0, 1: 1645.1. Samples: 20038106. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-12 21:37:36,443][43579] Avg episode reward: [(0, '263.750'), (1, '275.910')] [2023-10-12 21:37:36,606][44958] Updated weights for policy 0, policy_version 39040 (0.0007) [2023-10-12 21:37:38,101][44959] Updated weights for policy 1, policy_version 39240 (0.0010) [2023-10-12 21:37:38,465][44959] Updated weights for policy 1, policy_version 39250 (0.0009) [2023-10-12 21:37:38,837][44959] Updated weights for policy 1, policy_version 39260 (0.0010) [2023-10-12 21:37:40,829][44958] Updated weights for policy 0, policy_version 39050 (0.0007) [2023-10-12 21:37:41,211][44958] Updated weights for policy 0, policy_version 39060 (0.0007) [2023-10-12 21:37:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80183296. Throughput: 0: 1643.4, 1: 1656.5. Samples: 20058424. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-12 21:37:41,443][43579] Avg episode reward: [(0, '261.040'), (1, '279.210')] [2023-10-12 21:37:41,590][44958] Updated weights for policy 0, policy_version 39070 (0.0009) [2023-10-12 21:37:43,053][44959] Updated weights for policy 1, policy_version 39270 (0.0008) [2023-10-12 21:37:43,419][44959] Updated weights for policy 1, policy_version 39280 (0.0007) [2023-10-12 21:37:43,776][44959] Updated weights for policy 1, policy_version 39290 (0.0008) [2023-10-12 21:37:45,702][44958] Updated weights for policy 0, policy_version 39080 (0.0007) [2023-10-12 21:37:46,079][44958] Updated weights for policy 0, policy_version 39090 (0.0007) [2023-10-12 21:37:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80248832. Throughput: 0: 1638.5, 1: 1651.5. Samples: 20077782. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-12 21:37:46,443][43579] Avg episode reward: [(0, '262.840'), (1, '282.880')] [2023-10-12 21:37:46,448][44958] Updated weights for policy 0, policy_version 39100 (0.0008) [2023-10-12 21:37:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000039296_40239104.pth... [2023-10-12 21:37:46,490][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000037760_38666240.pth [2023-10-12 21:37:46,594][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000039104_40042496.pth... [2023-10-12 21:37:46,633][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000037568_38469632.pth [2023-10-12 21:37:47,881][44959] Updated weights for policy 1, policy_version 39300 (0.0008) [2023-10-12 21:37:48,250][44959] Updated weights for policy 1, policy_version 39310 (0.0008) [2023-10-12 21:37:48,618][44959] Updated weights for policy 1, policy_version 39320 (0.0009) [2023-10-12 21:37:50,702][44958] Updated weights for policy 0, policy_version 39110 (0.0008) [2023-10-12 21:37:51,074][44958] Updated weights for policy 0, policy_version 39120 (0.0009) [2023-10-12 21:37:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80314368. Throughput: 0: 1638.0, 1: 1644.3. Samples: 20087274. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:37:51,444][43579] Avg episode reward: [(0, '260.030'), (1, '277.460')] [2023-10-12 21:37:51,446][44958] Updated weights for policy 0, policy_version 39130 (0.0009) [2023-10-12 21:37:52,583][44959] Updated weights for policy 1, policy_version 39330 (0.0009) [2023-10-12 21:37:52,945][44959] Updated weights for policy 1, policy_version 39340 (0.0007) [2023-10-12 21:37:53,309][44959] Updated weights for policy 1, policy_version 39350 (0.0007) [2023-10-12 21:37:53,677][44959] Updated weights for policy 1, policy_version 39360 (0.0008) [2023-10-12 21:37:55,687][44958] Updated weights for policy 0, policy_version 39140 (0.0008) [2023-10-12 21:37:56,060][44958] Updated weights for policy 0, policy_version 39150 (0.0009) [2023-10-12 21:37:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80379904. Throughput: 0: 1638.3, 1: 1654.7. Samples: 20107998. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:37:56,443][44958] Updated weights for policy 0, policy_version 39160 (0.0009) [2023-10-12 21:37:56,443][43579] Avg episode reward: [(0, '265.520'), (1, '278.830')] [2023-10-12 21:37:57,935][44959] Updated weights for policy 1, policy_version 39370 (0.0008) [2023-10-12 21:37:58,300][44959] Updated weights for policy 1, policy_version 39380 (0.0009) [2023-10-12 21:37:58,664][44959] Updated weights for policy 1, policy_version 39390 (0.0009) [2023-10-12 21:38:00,573][44958] Updated weights for policy 0, policy_version 39170 (0.0008) [2023-10-12 21:38:00,942][44958] Updated weights for policy 0, policy_version 39180 (0.0009) [2023-10-12 21:38:01,314][44958] Updated weights for policy 0, policy_version 39190 (0.0010) [2023-10-12 21:38:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80445440. Throughput: 0: 1635.2, 1: 1647.4. Samples: 20127166. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:38:01,443][43579] Avg episode reward: [(0, '266.300'), (1, '278.350')] [2023-10-12 21:38:01,683][44958] Updated weights for policy 0, policy_version 39200 (0.0007) [2023-10-12 21:38:02,886][44959] Updated weights for policy 1, policy_version 39400 (0.0007) [2023-10-12 21:38:03,253][44959] Updated weights for policy 1, policy_version 39410 (0.0007) [2023-10-12 21:38:03,622][44959] Updated weights for policy 1, policy_version 39420 (0.0008) [2023-10-12 21:38:05,771][44958] Updated weights for policy 0, policy_version 39210 (0.0009) [2023-10-12 21:38:06,139][44958] Updated weights for policy 0, policy_version 39220 (0.0008) [2023-10-12 21:38:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80510976. Throughput: 0: 1635.0, 1: 1647.7. Samples: 20136904. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:38:06,443][43579] Avg episode reward: [(0, '269.450'), (1, '275.760')] [2023-10-12 21:38:06,507][44958] Updated weights for policy 0, policy_version 39230 (0.0007) [2023-10-12 21:38:07,744][44959] Updated weights for policy 1, policy_version 39430 (0.0007) [2023-10-12 21:38:08,103][44959] Updated weights for policy 1, policy_version 39440 (0.0008) [2023-10-12 21:38:08,475][44959] Updated weights for policy 1, policy_version 39450 (0.0010) [2023-10-12 21:38:10,680][44958] Updated weights for policy 0, policy_version 39240 (0.0008) [2023-10-12 21:38:11,061][44958] Updated weights for policy 0, policy_version 39250 (0.0009) [2023-10-12 21:38:11,430][44958] Updated weights for policy 0, policy_version 39260 (0.0009) [2023-10-12 21:38:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80576512. Throughput: 0: 1646.4, 1: 1653.7. Samples: 20157570. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:38:11,443][43579] Avg episode reward: [(0, '271.210'), (1, '275.250')] [2023-10-12 21:38:12,641][44959] Updated weights for policy 1, policy_version 39460 (0.0011) [2023-10-12 21:38:13,011][44959] Updated weights for policy 1, policy_version 39470 (0.0008) [2023-10-12 21:38:13,370][44959] Updated weights for policy 1, policy_version 39480 (0.0010) [2023-10-12 21:38:15,633][44958] Updated weights for policy 0, policy_version 39270 (0.0007) [2023-10-12 21:38:16,007][44958] Updated weights for policy 0, policy_version 39280 (0.0008) [2023-10-12 21:38:16,375][44958] Updated weights for policy 0, policy_version 39290 (0.0007) [2023-10-12 21:38:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80642048. Throughput: 0: 1643.9, 1: 1660.3. Samples: 20176966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:38:16,443][43579] Avg episode reward: [(0, '272.440'), (1, '273.390')] [2023-10-12 21:38:17,494][44959] Updated weights for policy 1, policy_version 39490 (0.0010) [2023-10-12 21:38:17,903][44959] Updated weights for policy 1, policy_version 39500 (0.0008) [2023-10-12 21:38:18,270][44959] Updated weights for policy 1, policy_version 39510 (0.0009) [2023-10-12 21:38:18,642][44959] Updated weights for policy 1, policy_version 39520 (0.0008) [2023-10-12 21:38:20,372][44958] Updated weights for policy 0, policy_version 39300 (0.0008) [2023-10-12 21:38:20,747][44958] Updated weights for policy 0, policy_version 39310 (0.0009) [2023-10-12 21:38:21,112][44958] Updated weights for policy 0, policy_version 39320 (0.0010) [2023-10-12 21:38:21,443][43579] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 80740352. Throughput: 0: 1647.7, 1: 1652.9. Samples: 20186636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:38:21,444][43579] Avg episode reward: [(0, '272.210'), (1, '280.080')] [2023-10-12 21:38:22,847][44959] Updated weights for policy 1, policy_version 39530 (0.0008) [2023-10-12 21:38:23,219][44959] Updated weights for policy 1, policy_version 39540 (0.0007) [2023-10-12 21:38:23,591][44959] Updated weights for policy 1, policy_version 39550 (0.0008) [2023-10-12 21:38:25,310][44958] Updated weights for policy 0, policy_version 39330 (0.0011) [2023-10-12 21:38:25,689][44958] Updated weights for policy 0, policy_version 39340 (0.0008) [2023-10-12 21:38:26,060][44958] Updated weights for policy 0, policy_version 39350 (0.0007) [2023-10-12 21:38:26,433][44958] Updated weights for policy 0, policy_version 39360 (0.0007) [2023-10-12 21:38:26,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 80805888. Throughput: 0: 1647.0, 1: 1651.2. Samples: 20206846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:38:26,444][43579] Avg episode reward: [(0, '270.940'), (1, '274.370')] [2023-10-12 21:38:27,763][44959] Updated weights for policy 1, policy_version 39560 (0.0010) [2023-10-12 21:38:28,135][44959] Updated weights for policy 1, policy_version 39570 (0.0008) [2023-10-12 21:38:28,511][44959] Updated weights for policy 1, policy_version 39580 (0.0010) [2023-10-12 21:38:30,841][44958] Updated weights for policy 0, policy_version 39370 (0.0007) [2023-10-12 21:38:31,219][44958] Updated weights for policy 0, policy_version 39380 (0.0009) [2023-10-12 21:38:31,443][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80838656. Throughput: 0: 1646.8, 1: 1653.4. Samples: 20226290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:38:31,444][43579] Avg episode reward: [(0, '271.930'), (1, '274.600')] [2023-10-12 21:38:31,598][44958] Updated weights for policy 0, policy_version 39390 (0.0009) [2023-10-12 21:38:32,560][44959] Updated weights for policy 1, policy_version 39590 (0.0009) [2023-10-12 21:38:32,940][44959] Updated weights for policy 1, policy_version 39600 (0.0011) [2023-10-12 21:38:33,304][44959] Updated weights for policy 1, policy_version 39610 (0.0009) [2023-10-12 21:38:35,692][44958] Updated weights for policy 0, policy_version 39400 (0.0008) [2023-10-12 21:38:36,066][44958] Updated weights for policy 0, policy_version 39410 (0.0009) [2023-10-12 21:38:36,442][44958] Updated weights for policy 0, policy_version 39420 (0.0010) [2023-10-12 21:38:36,442][43579] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80904192. Throughput: 0: 1648.7, 1: 1651.7. Samples: 20235788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:38:36,443][43579] Avg episode reward: [(0, '268.670'), (1, '276.170')] [2023-10-12 21:38:37,414][44959] Updated weights for policy 1, policy_version 39620 (0.0009) [2023-10-12 21:38:37,791][44959] Updated weights for policy 1, policy_version 39630 (0.0007) [2023-10-12 21:38:38,146][44959] Updated weights for policy 1, policy_version 39640 (0.0007) [2023-10-12 21:38:40,524][44958] Updated weights for policy 0, policy_version 39430 (0.0008) [2023-10-12 21:38:40,902][44958] Updated weights for policy 0, policy_version 39440 (0.0010) [2023-10-12 21:38:41,283][44958] Updated weights for policy 0, policy_version 39450 (0.0009) [2023-10-12 21:38:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 80969728. Throughput: 0: 1648.3, 1: 1645.3. Samples: 20256208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:38:41,444][43579] Avg episode reward: [(0, '271.080'), (1, '276.840')] [2023-10-12 21:38:42,307][44959] Updated weights for policy 1, policy_version 39650 (0.0007) [2023-10-12 21:38:42,666][44959] Updated weights for policy 1, policy_version 39660 (0.0008) [2023-10-12 21:38:43,040][44959] Updated weights for policy 1, policy_version 39670 (0.0009) [2023-10-12 21:38:43,410][44959] Updated weights for policy 1, policy_version 39680 (0.0008) [2023-10-12 21:38:45,503][44958] Updated weights for policy 0, policy_version 39460 (0.0009) [2023-10-12 21:38:45,870][44958] Updated weights for policy 0, policy_version 39470 (0.0009) [2023-10-12 21:38:46,246][44958] Updated weights for policy 0, policy_version 39480 (0.0007) [2023-10-12 21:38:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 81035264. Throughput: 0: 1646.2, 1: 1659.3. Samples: 20275916. Policy #0 lag: (min: 3.0, avg: 10.8, max: 35.0) [2023-10-12 21:38:46,443][43579] Avg episode reward: [(0, '273.630'), (1, '277.640')] [2023-10-12 21:38:47,380][44959] Updated weights for policy 1, policy_version 39690 (0.0010) [2023-10-12 21:38:47,747][44959] Updated weights for policy 1, policy_version 39700 (0.0007) [2023-10-12 21:38:48,125][44959] Updated weights for policy 1, policy_version 39710 (0.0008) [2023-10-12 21:38:50,352][44958] Updated weights for policy 0, policy_version 39490 (0.0008) [2023-10-12 21:38:50,731][44958] Updated weights for policy 0, policy_version 39500 (0.0009) [2023-10-12 21:38:51,106][44958] Updated weights for policy 0, policy_version 39510 (0.0009) [2023-10-12 21:38:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 81100800. Throughput: 0: 1650.0, 1: 1652.4. Samples: 20285512. Policy #0 lag: (min: 3.0, avg: 10.8, max: 35.0) [2023-10-12 21:38:51,443][43579] Avg episode reward: [(0, '272.660'), (1, '274.860')] [2023-10-12 21:38:51,476][44958] Updated weights for policy 0, policy_version 39520 (0.0008) [2023-10-12 21:38:52,341][44959] Updated weights for policy 1, policy_version 39720 (0.0011) [2023-10-12 21:38:52,707][44959] Updated weights for policy 1, policy_version 39730 (0.0009) [2023-10-12 21:38:53,084][44959] Updated weights for policy 1, policy_version 39740 (0.0011) [2023-10-12 21:38:55,685][44958] Updated weights for policy 0, policy_version 39530 (0.0007) [2023-10-12 21:38:56,062][44958] Updated weights for policy 0, policy_version 39540 (0.0007) [2023-10-12 21:38:56,433][44958] Updated weights for policy 0, policy_version 39550 (0.0009) [2023-10-12 21:38:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 81166336. Throughput: 0: 1641.5, 1: 1647.6. Samples: 20305582. Policy #0 lag: (min: 3.0, avg: 10.8, max: 35.0) [2023-10-12 21:38:56,444][43579] Avg episode reward: [(0, '273.650'), (1, '277.420')] [2023-10-12 21:38:57,332][44959] Updated weights for policy 1, policy_version 39750 (0.0010) [2023-10-12 21:38:57,703][44959] Updated weights for policy 1, policy_version 39760 (0.0011) [2023-10-12 21:38:58,074][44959] Updated weights for policy 1, policy_version 39770 (0.0011) [2023-10-12 21:39:00,655][44958] Updated weights for policy 0, policy_version 39560 (0.0010) [2023-10-12 21:39:01,044][44958] Updated weights for policy 0, policy_version 39570 (0.0009) [2023-10-12 21:39:01,412][44958] Updated weights for policy 0, policy_version 39580 (0.0009) [2023-10-12 21:39:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 81231872. Throughput: 0: 1636.3, 1: 1649.9. Samples: 20324844. Policy #0 lag: (min: 3.0, avg: 10.8, max: 35.0) [2023-10-12 21:39:01,443][43579] Avg episode reward: [(0, '265.430'), (1, '277.740')] [2023-10-12 21:39:02,258][44959] Updated weights for policy 1, policy_version 39780 (0.0010) [2023-10-12 21:39:02,663][44959] Updated weights for policy 1, policy_version 39790 (0.0008) [2023-10-12 21:39:03,029][44959] Updated weights for policy 1, policy_version 39800 (0.0007) [2023-10-12 21:39:05,463][44958] Updated weights for policy 0, policy_version 39590 (0.0008) [2023-10-12 21:39:05,834][44958] Updated weights for policy 0, policy_version 39600 (0.0007) [2023-10-12 21:39:06,198][44958] Updated weights for policy 0, policy_version 39610 (0.0007) [2023-10-12 21:39:06,443][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81330176. Throughput: 0: 1635.9, 1: 1644.8. Samples: 20334266. Policy #0 lag: (min: 3.0, avg: 10.8, max: 35.0) [2023-10-12 21:39:06,444][43579] Avg episode reward: [(0, '266.280'), (1, '270.690')] [2023-10-12 21:39:07,464][44959] Updated weights for policy 1, policy_version 39810 (0.0009) [2023-10-12 21:39:07,845][44959] Updated weights for policy 1, policy_version 39820 (0.0007) [2023-10-12 21:39:08,215][44959] Updated weights for policy 1, policy_version 39830 (0.0008) [2023-10-12 21:39:08,589][44959] Updated weights for policy 1, policy_version 39840 (0.0007) [2023-10-12 21:39:10,187][44958] Updated weights for policy 0, policy_version 39620 (0.0008) [2023-10-12 21:39:10,573][44958] Updated weights for policy 0, policy_version 39630 (0.0007) [2023-10-12 21:39:10,941][44958] Updated weights for policy 0, policy_version 39640 (0.0007) [2023-10-12 21:39:11,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81395712. Throughput: 0: 1637.6, 1: 1645.5. Samples: 20354586. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-12 21:39:11,444][43579] Avg episode reward: [(0, '260.210'), (1, '271.260')] [2023-10-12 21:39:12,564][44959] Updated weights for policy 1, policy_version 39850 (0.0010) [2023-10-12 21:39:12,937][44959] Updated weights for policy 1, policy_version 39860 (0.0008) [2023-10-12 21:39:13,299][44959] Updated weights for policy 1, policy_version 39870 (0.0009) [2023-10-12 21:39:15,123][44958] Updated weights for policy 0, policy_version 39650 (0.0007) [2023-10-12 21:39:15,497][44958] Updated weights for policy 0, policy_version 39660 (0.0007) [2023-10-12 21:39:15,875][44958] Updated weights for policy 0, policy_version 39670 (0.0007) [2023-10-12 21:39:16,248][44958] Updated weights for policy 0, policy_version 39680 (0.0009) [2023-10-12 21:39:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81461248. Throughput: 0: 1637.6, 1: 1648.3. Samples: 20374152. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-12 21:39:16,443][43579] Avg episode reward: [(0, '263.800'), (1, '272.030')] [2023-10-12 21:39:17,383][44959] Updated weights for policy 1, policy_version 39880 (0.0009) [2023-10-12 21:39:17,745][44959] Updated weights for policy 1, policy_version 39890 (0.0011) [2023-10-12 21:39:18,123][44959] Updated weights for policy 1, policy_version 39900 (0.0008) [2023-10-12 21:39:20,477][44958] Updated weights for policy 0, policy_version 39690 (0.0008) [2023-10-12 21:39:20,847][44958] Updated weights for policy 0, policy_version 39700 (0.0007) [2023-10-12 21:39:21,226][44958] Updated weights for policy 0, policy_version 39710 (0.0010) [2023-10-12 21:39:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 81526784. Throughput: 0: 1644.8, 1: 1651.8. Samples: 20384136. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-12 21:39:21,444][43579] Avg episode reward: [(0, '265.140'), (1, '268.860')] [2023-10-12 21:39:22,359][44959] Updated weights for policy 1, policy_version 39910 (0.0008) [2023-10-12 21:39:22,722][44959] Updated weights for policy 1, policy_version 39920 (0.0007) [2023-10-12 21:39:23,090][44959] Updated weights for policy 1, policy_version 39930 (0.0007) [2023-10-12 21:39:25,488][44958] Updated weights for policy 0, policy_version 39720 (0.0009) [2023-10-12 21:39:25,859][44958] Updated weights for policy 0, policy_version 39730 (0.0009) [2023-10-12 21:39:26,235][44958] Updated weights for policy 0, policy_version 39740 (0.0009) [2023-10-12 21:39:26,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 81592320. Throughput: 0: 1641.6, 1: 1651.7. Samples: 20404406. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-12 21:39:26,443][43579] Avg episode reward: [(0, '274.800'), (1, '271.420')] [2023-10-12 21:39:27,454][44959] Updated weights for policy 1, policy_version 39940 (0.0007) [2023-10-12 21:39:27,821][44959] Updated weights for policy 1, policy_version 39950 (0.0008) [2023-10-12 21:39:28,189][44959] Updated weights for policy 1, policy_version 39960 (0.0008) [2023-10-12 21:39:30,394][44958] Updated weights for policy 0, policy_version 39750 (0.0008) [2023-10-12 21:39:30,768][44958] Updated weights for policy 0, policy_version 39760 (0.0009) [2023-10-12 21:39:31,144][44958] Updated weights for policy 0, policy_version 39770 (0.0009) [2023-10-12 21:39:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81657856. Throughput: 0: 1642.8, 1: 1640.7. Samples: 20423676. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-12 21:39:31,444][43579] Avg episode reward: [(0, '268.230'), (1, '268.100')] [2023-10-12 21:39:32,148][44959] Updated weights for policy 1, policy_version 39970 (0.0010) [2023-10-12 21:39:32,509][44959] Updated weights for policy 1, policy_version 39980 (0.0009) [2023-10-12 21:39:32,876][44959] Updated weights for policy 1, policy_version 39990 (0.0010) [2023-10-12 21:39:33,248][44959] Updated weights for policy 1, policy_version 40000 (0.0010) [2023-10-12 21:39:35,181][44958] Updated weights for policy 0, policy_version 39780 (0.0009) [2023-10-12 21:39:35,556][44958] Updated weights for policy 0, policy_version 39790 (0.0009) [2023-10-12 21:39:35,933][44958] Updated weights for policy 0, policy_version 39800 (0.0009) [2023-10-12 21:39:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81723392. Throughput: 0: 1649.6, 1: 1644.0. Samples: 20433724. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-12 21:39:36,444][43579] Avg episode reward: [(0, '271.100'), (1, '268.590')] [2023-10-12 21:39:37,479][44959] Updated weights for policy 1, policy_version 40010 (0.0008) [2023-10-12 21:39:37,852][44959] Updated weights for policy 1, policy_version 40020 (0.0007) [2023-10-12 21:39:38,222][44959] Updated weights for policy 1, policy_version 40030 (0.0009) [2023-10-12 21:39:40,151][44958] Updated weights for policy 0, policy_version 39810 (0.0009) [2023-10-12 21:39:40,528][44958] Updated weights for policy 0, policy_version 39820 (0.0009) [2023-10-12 21:39:40,900][44958] Updated weights for policy 0, policy_version 39830 (0.0009) [2023-10-12 21:39:41,273][44958] Updated weights for policy 0, policy_version 39840 (0.0009) [2023-10-12 21:39:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 81788928. Throughput: 0: 1646.5, 1: 1654.7. Samples: 20454138. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:39:41,443][43579] Avg episode reward: [(0, '268.620'), (1, '266.450')] [2023-10-12 21:39:42,341][44959] Updated weights for policy 1, policy_version 40040 (0.0009) [2023-10-12 21:39:42,706][44959] Updated weights for policy 1, policy_version 40050 (0.0007) [2023-10-12 21:39:43,077][44959] Updated weights for policy 1, policy_version 40060 (0.0008) [2023-10-12 21:39:45,498][44958] Updated weights for policy 0, policy_version 39850 (0.0007) [2023-10-12 21:39:45,865][44958] Updated weights for policy 0, policy_version 39860 (0.0008) [2023-10-12 21:39:46,235][44958] Updated weights for policy 0, policy_version 39870 (0.0009) [2023-10-12 21:39:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81854464. Throughput: 0: 1651.2, 1: 1652.0. Samples: 20473490. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:39:46,443][43579] Avg episode reward: [(0, '267.650'), (1, '263.060')] [2023-10-12 21:39:46,450][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000040064_41025536.pth... [2023-10-12 21:39:46,450][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000039872_40828928.pth... [2023-10-12 21:39:46,480][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000038528_39452672.pth [2023-10-12 21:39:46,489][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000038336_39256064.pth [2023-10-12 21:39:47,386][44959] Updated weights for policy 1, policy_version 40070 (0.0009) [2023-10-12 21:39:47,774][44959] Updated weights for policy 1, policy_version 40080 (0.0007) [2023-10-12 21:39:48,138][44959] Updated weights for policy 1, policy_version 40090 (0.0009) [2023-10-12 21:39:50,399][44958] Updated weights for policy 0, policy_version 39880 (0.0009) [2023-10-12 21:39:50,775][44958] Updated weights for policy 0, policy_version 39890 (0.0007) [2023-10-12 21:39:51,143][44958] Updated weights for policy 0, policy_version 39900 (0.0007) [2023-10-12 21:39:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81920000. Throughput: 0: 1657.0, 1: 1653.5. Samples: 20483240. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:39:51,444][43579] Avg episode reward: [(0, '270.130'), (1, '261.990')] [2023-10-12 21:39:52,054][44959] Updated weights for policy 1, policy_version 40100 (0.0008) [2023-10-12 21:39:52,426][44959] Updated weights for policy 1, policy_version 40110 (0.0008) [2023-10-12 21:39:52,793][44959] Updated weights for policy 1, policy_version 40120 (0.0009) [2023-10-12 21:39:55,313][44958] Updated weights for policy 0, policy_version 39910 (0.0007) [2023-10-12 21:39:55,694][44958] Updated weights for policy 0, policy_version 39920 (0.0008) [2023-10-12 21:39:56,069][44958] Updated weights for policy 0, policy_version 39930 (0.0007) [2023-10-12 21:39:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 81985536. Throughput: 0: 1650.1, 1: 1657.2. Samples: 20503416. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:39:56,443][43579] Avg episode reward: [(0, '272.240'), (1, '265.650')] [2023-10-12 21:39:56,975][44959] Updated weights for policy 1, policy_version 40130 (0.0008) [2023-10-12 21:39:57,346][44959] Updated weights for policy 1, policy_version 40140 (0.0010) [2023-10-12 21:39:57,715][44959] Updated weights for policy 1, policy_version 40150 (0.0008) [2023-10-12 21:39:58,080][44959] Updated weights for policy 1, policy_version 40160 (0.0008) [2023-10-12 21:40:00,198][44958] Updated weights for policy 0, policy_version 39940 (0.0009) [2023-10-12 21:40:00,577][44958] Updated weights for policy 0, policy_version 39950 (0.0008) [2023-10-12 21:40:00,947][44958] Updated weights for policy 0, policy_version 39960 (0.0008) [2023-10-12 21:40:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 82051072. Throughput: 0: 1647.3, 1: 1657.1. Samples: 20522850. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:40:01,444][43579] Avg episode reward: [(0, '271.120'), (1, '273.060')] [2023-10-12 21:40:02,147][44959] Updated weights for policy 1, policy_version 40170 (0.0010) [2023-10-12 21:40:02,506][44959] Updated weights for policy 1, policy_version 40180 (0.0007) [2023-10-12 21:40:02,882][44959] Updated weights for policy 1, policy_version 40190 (0.0007) [2023-10-12 21:40:05,306][44958] Updated weights for policy 0, policy_version 39970 (0.0008) [2023-10-12 21:40:05,675][44958] Updated weights for policy 0, policy_version 39980 (0.0008) [2023-10-12 21:40:06,051][44958] Updated weights for policy 0, policy_version 39990 (0.0008) [2023-10-12 21:40:06,421][44958] Updated weights for policy 0, policy_version 40000 (0.0007) [2023-10-12 21:40:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82116608. Throughput: 0: 1642.0, 1: 1659.0. Samples: 20532680. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-12 21:40:06,444][43579] Avg episode reward: [(0, '275.690'), (1, '270.100')] [2023-10-12 21:40:07,085][44959] Updated weights for policy 1, policy_version 40200 (0.0008) [2023-10-12 21:40:07,455][44959] Updated weights for policy 1, policy_version 40210 (0.0007) [2023-10-12 21:40:07,821][44959] Updated weights for policy 1, policy_version 40220 (0.0007) [2023-10-12 21:40:10,448][44958] Updated weights for policy 0, policy_version 40010 (0.0007) [2023-10-12 21:40:10,828][44958] Updated weights for policy 0, policy_version 40020 (0.0007) [2023-10-12 21:40:11,209][44958] Updated weights for policy 0, policy_version 40030 (0.0007) [2023-10-12 21:40:11,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82182144. Throughput: 0: 1645.6, 1: 1657.2. Samples: 20553032. Policy #0 lag: (min: 20.0, avg: 22.4, max: 52.0) [2023-10-12 21:40:11,444][43579] Avg episode reward: [(0, '275.750'), (1, '273.540')] [2023-10-12 21:40:11,821][44959] Updated weights for policy 1, policy_version 40230 (0.0009) [2023-10-12 21:40:12,182][44959] Updated weights for policy 1, policy_version 40240 (0.0007) [2023-10-12 21:40:12,553][44959] Updated weights for policy 1, policy_version 40250 (0.0009) [2023-10-12 21:40:15,303][44958] Updated weights for policy 0, policy_version 40040 (0.0008) [2023-10-12 21:40:15,680][44958] Updated weights for policy 0, policy_version 40050 (0.0007) [2023-10-12 21:40:16,065][44958] Updated weights for policy 0, policy_version 40060 (0.0008) [2023-10-12 21:40:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82247680. Throughput: 0: 1644.4, 1: 1661.6. Samples: 20572444. Policy #0 lag: (min: 20.0, avg: 22.4, max: 52.0) [2023-10-12 21:40:16,443][43579] Avg episode reward: [(0, '276.810'), (1, '280.630')] [2023-10-12 21:40:16,782][44959] Updated weights for policy 1, policy_version 40260 (0.0009) [2023-10-12 21:40:17,151][44959] Updated weights for policy 1, policy_version 40270 (0.0009) [2023-10-12 21:40:17,521][44959] Updated weights for policy 1, policy_version 40280 (0.0008) [2023-10-12 21:40:20,343][44958] Updated weights for policy 0, policy_version 40070 (0.0008) [2023-10-12 21:40:20,713][44958] Updated weights for policy 0, policy_version 40080 (0.0007) [2023-10-12 21:40:21,093][44958] Updated weights for policy 0, policy_version 40090 (0.0007) [2023-10-12 21:40:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82313216. Throughput: 0: 1646.1, 1: 1658.8. Samples: 20582444. Policy #0 lag: (min: 20.0, avg: 22.4, max: 52.0) [2023-10-12 21:40:21,443][43579] Avg episode reward: [(0, '275.080'), (1, '280.850')] [2023-10-12 21:40:21,728][44959] Updated weights for policy 1, policy_version 40290 (0.0009) [2023-10-12 21:40:22,091][44959] Updated weights for policy 1, policy_version 40300 (0.0010) [2023-10-12 21:40:22,468][44959] Updated weights for policy 1, policy_version 40310 (0.0009) [2023-10-12 21:40:22,828][44959] Updated weights for policy 1, policy_version 40320 (0.0009) [2023-10-12 21:40:25,169][44958] Updated weights for policy 0, policy_version 40100 (0.0007) [2023-10-12 21:40:25,546][44958] Updated weights for policy 0, policy_version 40110 (0.0010) [2023-10-12 21:40:25,922][44958] Updated weights for policy 0, policy_version 40120 (0.0008) [2023-10-12 21:40:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82378752. Throughput: 0: 1646.0, 1: 1649.5. Samples: 20602434. Policy #0 lag: (min: 20.0, avg: 22.4, max: 52.0) [2023-10-12 21:40:26,443][43579] Avg episode reward: [(0, '275.980'), (1, '277.530')] [2023-10-12 21:40:26,933][44959] Updated weights for policy 1, policy_version 40330 (0.0008) [2023-10-12 21:40:27,303][44959] Updated weights for policy 1, policy_version 40340 (0.0007) [2023-10-12 21:40:27,673][44959] Updated weights for policy 1, policy_version 40350 (0.0008) [2023-10-12 21:40:29,943][44958] Updated weights for policy 0, policy_version 40130 (0.0007) [2023-10-12 21:40:30,317][44958] Updated weights for policy 0, policy_version 40140 (0.0008) [2023-10-12 21:40:30,677][44958] Updated weights for policy 0, policy_version 40150 (0.0007) [2023-10-12 21:40:31,056][44958] Updated weights for policy 0, policy_version 40160 (0.0007) [2023-10-12 21:40:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82444288. Throughput: 0: 1645.5, 1: 1651.0. Samples: 20621832. Policy #0 lag: (min: 20.0, avg: 22.4, max: 52.0) [2023-10-12 21:40:31,443][43579] Avg episode reward: [(0, '272.920'), (1, '275.210')] [2023-10-12 21:40:31,751][44959] Updated weights for policy 1, policy_version 40360 (0.0008) [2023-10-12 21:40:32,120][44959] Updated weights for policy 1, policy_version 40370 (0.0008) [2023-10-12 21:40:32,494][44959] Updated weights for policy 1, policy_version 40380 (0.0008) [2023-10-12 21:40:35,337][44958] Updated weights for policy 0, policy_version 40170 (0.0008) [2023-10-12 21:40:35,701][44958] Updated weights for policy 0, policy_version 40180 (0.0007) [2023-10-12 21:40:36,086][44958] Updated weights for policy 0, policy_version 40190 (0.0008) [2023-10-12 21:40:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82509824. Throughput: 0: 1648.8, 1: 1657.3. Samples: 20632014. Policy #0 lag: (min: 20.0, avg: 22.4, max: 52.0) [2023-10-12 21:40:36,444][43579] Avg episode reward: [(0, '270.020'), (1, '278.420')] [2023-10-12 21:40:36,705][44959] Updated weights for policy 1, policy_version 40390 (0.0008) [2023-10-12 21:40:37,073][44959] Updated weights for policy 1, policy_version 40400 (0.0008) [2023-10-12 21:40:37,449][44959] Updated weights for policy 1, policy_version 40410 (0.0007) [2023-10-12 21:40:40,092][44958] Updated weights for policy 0, policy_version 40200 (0.0008) [2023-10-12 21:40:40,482][44958] Updated weights for policy 0, policy_version 40210 (0.0009) [2023-10-12 21:40:40,846][44958] Updated weights for policy 0, policy_version 40220 (0.0008) [2023-10-12 21:40:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82575360. Throughput: 0: 1639.8, 1: 1650.6. Samples: 20651486. Policy #0 lag: (min: 9.0, avg: 11.4, max: 39.0) [2023-10-12 21:40:41,444][43579] Avg episode reward: [(0, '271.260'), (1, '275.400')] [2023-10-12 21:40:41,544][44959] Updated weights for policy 1, policy_version 40420 (0.0007) [2023-10-12 21:40:41,906][44959] Updated weights for policy 1, policy_version 40430 (0.0007) [2023-10-12 21:40:42,271][44959] Updated weights for policy 1, policy_version 40440 (0.0007) [2023-10-12 21:40:45,154][44958] Updated weights for policy 0, policy_version 40230 (0.0008) [2023-10-12 21:40:45,526][44958] Updated weights for policy 0, policy_version 40240 (0.0009) [2023-10-12 21:40:45,899][44958] Updated weights for policy 0, policy_version 40250 (0.0008) [2023-10-12 21:40:46,326][44959] Updated weights for policy 1, policy_version 40450 (0.0008) [2023-10-12 21:40:46,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82640896. Throughput: 0: 1643.1, 1: 1654.2. Samples: 20671228. Policy #0 lag: (min: 9.0, avg: 11.4, max: 39.0) [2023-10-12 21:40:46,443][43579] Avg episode reward: [(0, '275.550'), (1, '276.990')] [2023-10-12 21:40:46,700][44959] Updated weights for policy 1, policy_version 40460 (0.0007) [2023-10-12 21:40:47,068][44959] Updated weights for policy 1, policy_version 40470 (0.0008) [2023-10-12 21:40:47,434][44959] Updated weights for policy 1, policy_version 40480 (0.0008) [2023-10-12 21:40:50,285][44958] Updated weights for policy 0, policy_version 40260 (0.0008) [2023-10-12 21:40:50,647][44958] Updated weights for policy 0, policy_version 40270 (0.0007) [2023-10-12 21:40:51,027][44958] Updated weights for policy 0, policy_version 40280 (0.0007) [2023-10-12 21:40:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82706432. Throughput: 0: 1648.6, 1: 1648.6. Samples: 20681054. Policy #0 lag: (min: 9.0, avg: 11.4, max: 39.0) [2023-10-12 21:40:51,444][43579] Avg episode reward: [(0, '273.980'), (1, '277.460')] [2023-10-12 21:40:51,697][44959] Updated weights for policy 1, policy_version 40490 (0.0010) [2023-10-12 21:40:52,060][44959] Updated weights for policy 1, policy_version 40500 (0.0009) [2023-10-12 21:40:52,433][44959] Updated weights for policy 1, policy_version 40510 (0.0008) [2023-10-12 21:40:54,979][44958] Updated weights for policy 0, policy_version 40290 (0.0008) [2023-10-12 21:40:55,337][44958] Updated weights for policy 0, policy_version 40300 (0.0009) [2023-10-12 21:40:55,715][44958] Updated weights for policy 0, policy_version 40310 (0.0008) [2023-10-12 21:40:56,072][44958] Updated weights for policy 0, policy_version 40320 (0.0009) [2023-10-12 21:40:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82771968. Throughput: 0: 1645.2, 1: 1646.9. Samples: 20701174. Policy #0 lag: (min: 9.0, avg: 11.4, max: 39.0) [2023-10-12 21:40:56,444][43579] Avg episode reward: [(0, '269.250'), (1, '275.830')] [2023-10-12 21:40:56,666][44959] Updated weights for policy 1, policy_version 40520 (0.0009) [2023-10-12 21:40:57,041][44959] Updated weights for policy 1, policy_version 40530 (0.0011) [2023-10-12 21:40:57,402][44959] Updated weights for policy 1, policy_version 40540 (0.0011) [2023-10-12 21:41:00,303][44958] Updated weights for policy 0, policy_version 40330 (0.0008) [2023-10-12 21:41:00,667][44958] Updated weights for policy 0, policy_version 40340 (0.0011) [2023-10-12 21:41:01,052][44958] Updated weights for policy 0, policy_version 40350 (0.0009) [2023-10-12 21:41:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 82837504. Throughput: 0: 1638.9, 1: 1648.8. Samples: 20720392. Policy #0 lag: (min: 9.0, avg: 11.4, max: 39.0) [2023-10-12 21:41:01,443][43579] Avg episode reward: [(0, '272.890'), (1, '278.430')] [2023-10-12 21:41:01,486][44959] Updated weights for policy 1, policy_version 40550 (0.0009) [2023-10-12 21:41:01,848][44959] Updated weights for policy 1, policy_version 40560 (0.0011) [2023-10-12 21:41:02,221][44959] Updated weights for policy 1, policy_version 40570 (0.0008) [2023-10-12 21:41:05,258][44958] Updated weights for policy 0, policy_version 40360 (0.0008) [2023-10-12 21:41:05,637][44958] Updated weights for policy 0, policy_version 40370 (0.0007) [2023-10-12 21:41:06,006][44958] Updated weights for policy 0, policy_version 40380 (0.0007) [2023-10-12 21:41:06,344][44959] Updated weights for policy 1, policy_version 40580 (0.0009) [2023-10-12 21:41:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82903040. Throughput: 0: 1639.0, 1: 1649.8. Samples: 20730438. Policy #0 lag: (min: 9.0, avg: 11.4, max: 39.0) [2023-10-12 21:41:06,443][43579] Avg episode reward: [(0, '272.600'), (1, '277.130')] [2023-10-12 21:41:06,718][44959] Updated weights for policy 1, policy_version 40590 (0.0009) [2023-10-12 21:41:07,075][44959] Updated weights for policy 1, policy_version 40600 (0.0009) [2023-10-12 21:41:10,099][44958] Updated weights for policy 0, policy_version 40390 (0.0008) [2023-10-12 21:41:10,464][44958] Updated weights for policy 0, policy_version 40400 (0.0007) [2023-10-12 21:41:10,845][44958] Updated weights for policy 0, policy_version 40410 (0.0008) [2023-10-12 21:41:11,219][44959] Updated weights for policy 1, policy_version 40610 (0.0010) [2023-10-12 21:41:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 82968576. Throughput: 0: 1637.5, 1: 1653.0. Samples: 20750506. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:41:11,444][43579] Avg episode reward: [(0, '272.900'), (1, '281.820')] [2023-10-12 21:41:11,585][44959] Updated weights for policy 1, policy_version 40620 (0.0011) [2023-10-12 21:41:11,957][44959] Updated weights for policy 1, policy_version 40630 (0.0010) [2023-10-12 21:41:12,312][44959] Updated weights for policy 1, policy_version 40640 (0.0008) [2023-10-12 21:41:15,066][44958] Updated weights for policy 0, policy_version 40420 (0.0008) [2023-10-12 21:41:15,449][44958] Updated weights for policy 0, policy_version 40430 (0.0009) [2023-10-12 21:41:15,832][44958] Updated weights for policy 0, policy_version 40440 (0.0008) [2023-10-12 21:41:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 83034112. Throughput: 0: 1637.3, 1: 1654.1. Samples: 20769948. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:41:16,444][43579] Avg episode reward: [(0, '267.800'), (1, '280.050')] [2023-10-12 21:41:16,682][44959] Updated weights for policy 1, policy_version 40650 (0.0007) [2023-10-12 21:41:17,054][44959] Updated weights for policy 1, policy_version 40660 (0.0008) [2023-10-12 21:41:17,417][44959] Updated weights for policy 1, policy_version 40670 (0.0007) [2023-10-12 21:41:19,893][44958] Updated weights for policy 0, policy_version 40450 (0.0007) [2023-10-12 21:41:20,304][44958] Updated weights for policy 0, policy_version 40460 (0.0010) [2023-10-12 21:41:20,680][44958] Updated weights for policy 0, policy_version 40470 (0.0009) [2023-10-12 21:41:21,048][44958] Updated weights for policy 0, policy_version 40480 (0.0009) [2023-10-12 21:41:21,419][44959] Updated weights for policy 1, policy_version 40680 (0.0007) [2023-10-12 21:41:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83099648. Throughput: 0: 1637.2, 1: 1650.5. Samples: 20779960. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:41:21,443][43579] Avg episode reward: [(0, '274.790'), (1, '275.700')] [2023-10-12 21:41:21,791][44959] Updated weights for policy 1, policy_version 40690 (0.0008) [2023-10-12 21:41:22,169][44959] Updated weights for policy 1, policy_version 40700 (0.0010) [2023-10-12 21:41:25,264][44958] Updated weights for policy 0, policy_version 40490 (0.0011) [2023-10-12 21:41:25,640][44958] Updated weights for policy 0, policy_version 40500 (0.0011) [2023-10-12 21:41:26,012][44958] Updated weights for policy 0, policy_version 40510 (0.0009) [2023-10-12 21:41:26,249][44959] Updated weights for policy 1, policy_version 40710 (0.0009) [2023-10-12 21:41:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83165184. Throughput: 0: 1641.4, 1: 1657.9. Samples: 20799954. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:41:26,443][43579] Avg episode reward: [(0, '280.500'), (1, '280.210')] [2023-10-12 21:41:26,627][44959] Updated weights for policy 1, policy_version 40720 (0.0007) [2023-10-12 21:41:27,005][44959] Updated weights for policy 1, policy_version 40730 (0.0007) [2023-10-12 21:41:30,401][44958] Updated weights for policy 0, policy_version 40520 (0.0010) [2023-10-12 21:41:30,766][44958] Updated weights for policy 0, policy_version 40530 (0.0010) [2023-10-12 21:41:31,112][44959] Updated weights for policy 1, policy_version 40740 (0.0009) [2023-10-12 21:41:31,139][44958] Updated weights for policy 0, policy_version 40540 (0.0008) [2023-10-12 21:41:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83230720. Throughput: 0: 1636.8, 1: 1652.5. Samples: 20819248. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:41:31,443][43579] Avg episode reward: [(0, '280.870'), (1, '280.790')] [2023-10-12 21:41:31,476][44959] Updated weights for policy 1, policy_version 40750 (0.0010) [2023-10-12 21:41:31,834][44959] Updated weights for policy 1, policy_version 40760 (0.0009) [2023-10-12 21:41:35,243][44958] Updated weights for policy 0, policy_version 40550 (0.0008) [2023-10-12 21:41:35,610][44958] Updated weights for policy 0, policy_version 40560 (0.0007) [2023-10-12 21:41:35,984][44958] Updated weights for policy 0, policy_version 40570 (0.0008) [2023-10-12 21:41:36,014][44959] Updated weights for policy 1, policy_version 40770 (0.0010) [2023-10-12 21:41:36,377][44959] Updated weights for policy 1, policy_version 40780 (0.0010) [2023-10-12 21:41:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83296256. Throughput: 0: 1635.2, 1: 1658.8. Samples: 20829280. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:41:36,444][43579] Avg episode reward: [(0, '275.060'), (1, '278.310')] [2023-10-12 21:41:36,753][44959] Updated weights for policy 1, policy_version 40790 (0.0010) [2023-10-12 21:41:37,123][44959] Updated weights for policy 1, policy_version 40800 (0.0010) [2023-10-12 21:41:40,106][44958] Updated weights for policy 0, policy_version 40580 (0.0008) [2023-10-12 21:41:40,475][44958] Updated weights for policy 0, policy_version 40590 (0.0007) [2023-10-12 21:41:40,851][44958] Updated weights for policy 0, policy_version 40600 (0.0007) [2023-10-12 21:41:41,324][44959] Updated weights for policy 1, policy_version 40810 (0.0007) [2023-10-12 21:41:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83361792. Throughput: 0: 1631.7, 1: 1655.1. Samples: 20849078. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 21:41:41,443][43579] Avg episode reward: [(0, '274.060'), (1, '277.270')] [2023-10-12 21:41:41,700][44959] Updated weights for policy 1, policy_version 40820 (0.0008) [2023-10-12 21:41:42,056][44959] Updated weights for policy 1, policy_version 40830 (0.0008) [2023-10-12 21:41:44,792][44958] Updated weights for policy 0, policy_version 40610 (0.0008) [2023-10-12 21:41:45,167][44958] Updated weights for policy 0, policy_version 40620 (0.0007) [2023-10-12 21:41:45,543][44958] Updated weights for policy 0, policy_version 40630 (0.0008) [2023-10-12 21:41:45,906][44958] Updated weights for policy 0, policy_version 40640 (0.0010) [2023-10-12 21:41:46,295][44959] Updated weights for policy 1, policy_version 40840 (0.0007) [2023-10-12 21:41:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83427328. Throughput: 0: 1643.7, 1: 1648.3. Samples: 20868532. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 21:41:46,443][43579] Avg episode reward: [(0, '273.950'), (1, '276.610')] [2023-10-12 21:41:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000040640_41615360.pth... [2023-10-12 21:41:46,484][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000039104_40042496.pth [2023-10-12 21:41:46,657][44959] Updated weights for policy 1, policy_version 40850 (0.0007) [2023-10-12 21:41:47,032][44959] Updated weights for policy 1, policy_version 40860 (0.0008) [2023-10-12 21:41:47,178][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000040864_41844736.pth... [2023-10-12 21:41:47,219][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000039296_40239104.pth [2023-10-12 21:41:50,173][44958] Updated weights for policy 0, policy_version 40650 (0.0009) [2023-10-12 21:41:50,541][44958] Updated weights for policy 0, policy_version 40660 (0.0007) [2023-10-12 21:41:50,923][44958] Updated weights for policy 0, policy_version 40670 (0.0007) [2023-10-12 21:41:51,149][44959] Updated weights for policy 1, policy_version 40870 (0.0008) [2023-10-12 21:41:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83492864. Throughput: 0: 1641.8, 1: 1649.5. Samples: 20878546. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 21:41:51,443][43579] Avg episode reward: [(0, '270.620'), (1, '280.040')] [2023-10-12 21:41:51,511][44959] Updated weights for policy 1, policy_version 40880 (0.0009) [2023-10-12 21:41:51,886][44959] Updated weights for policy 1, policy_version 40890 (0.0007) [2023-10-12 21:41:55,112][44958] Updated weights for policy 0, policy_version 40680 (0.0007) [2023-10-12 21:41:55,479][44958] Updated weights for policy 0, policy_version 40690 (0.0008) [2023-10-12 21:41:55,857][44958] Updated weights for policy 0, policy_version 40700 (0.0009) [2023-10-12 21:41:55,998][44959] Updated weights for policy 1, policy_version 40900 (0.0008) [2023-10-12 21:41:56,375][44959] Updated weights for policy 1, policy_version 40910 (0.0009) [2023-10-12 21:41:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83558400. Throughput: 0: 1636.9, 1: 1651.5. Samples: 20898486. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 21:41:56,443][43579] Avg episode reward: [(0, '265.690'), (1, '280.870')] [2023-10-12 21:41:56,747][44959] Updated weights for policy 1, policy_version 40920 (0.0007) [2023-10-12 21:41:59,894][44958] Updated weights for policy 0, policy_version 40710 (0.0008) [2023-10-12 21:42:00,270][44958] Updated weights for policy 0, policy_version 40720 (0.0009) [2023-10-12 21:42:00,648][44958] Updated weights for policy 0, policy_version 40730 (0.0009) [2023-10-12 21:42:01,000][44959] Updated weights for policy 1, policy_version 40930 (0.0008) [2023-10-12 21:42:01,364][44959] Updated weights for policy 1, policy_version 40940 (0.0009) [2023-10-12 21:42:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83623936. Throughput: 0: 1636.3, 1: 1644.4. Samples: 20917576. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 21:42:01,443][43579] Avg episode reward: [(0, '269.250'), (1, '281.940')] [2023-10-12 21:42:01,734][44959] Updated weights for policy 1, policy_version 40950 (0.0008) [2023-10-12 21:42:02,105][44959] Updated weights for policy 1, policy_version 40960 (0.0007) [2023-10-12 21:42:04,911][44958] Updated weights for policy 0, policy_version 40740 (0.0009) [2023-10-12 21:42:05,297][44958] Updated weights for policy 0, policy_version 40750 (0.0008) [2023-10-12 21:42:05,665][44958] Updated weights for policy 0, policy_version 40760 (0.0007) [2023-10-12 21:42:06,440][44959] Updated weights for policy 1, policy_version 40970 (0.0009) [2023-10-12 21:42:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83689472. Throughput: 0: 1637.1, 1: 1645.6. Samples: 20927680. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 21:42:06,444][43579] Avg episode reward: [(0, '272.730'), (1, '285.560')] [2023-10-12 21:42:06,815][44959] Updated weights for policy 1, policy_version 40980 (0.0009) [2023-10-12 21:42:07,186][44959] Updated weights for policy 1, policy_version 40990 (0.0009) [2023-10-12 21:42:09,704][44958] Updated weights for policy 0, policy_version 40770 (0.0010) [2023-10-12 21:42:10,067][44958] Updated weights for policy 0, policy_version 40780 (0.0010) [2023-10-12 21:42:10,441][44958] Updated weights for policy 0, policy_version 40790 (0.0007) [2023-10-12 21:42:10,816][44958] Updated weights for policy 0, policy_version 40800 (0.0008) [2023-10-12 21:42:11,114][44959] Updated weights for policy 1, policy_version 41000 (0.0008) [2023-10-12 21:42:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83755008. Throughput: 0: 1632.2, 1: 1643.8. Samples: 20947372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:42:11,443][43579] Avg episode reward: [(0, '275.330'), (1, '288.320')] [2023-10-12 21:42:11,479][44959] Updated weights for policy 1, policy_version 41010 (0.0008) [2023-10-12 21:42:11,842][44959] Updated weights for policy 1, policy_version 41020 (0.0010) [2023-10-12 21:42:15,113][44958] Updated weights for policy 0, policy_version 40810 (0.0009) [2023-10-12 21:42:15,483][44958] Updated weights for policy 0, policy_version 40820 (0.0008) [2023-10-12 21:42:15,853][44958] Updated weights for policy 0, policy_version 40830 (0.0008) [2023-10-12 21:42:16,242][44959] Updated weights for policy 1, policy_version 41030 (0.0010) [2023-10-12 21:42:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 83820544. Throughput: 0: 1640.8, 1: 1640.2. Samples: 20966894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:42:16,443][43579] Avg episode reward: [(0, '278.300'), (1, '279.560')] [2023-10-12 21:42:16,602][44959] Updated weights for policy 1, policy_version 41040 (0.0008) [2023-10-12 21:42:16,969][44959] Updated weights for policy 1, policy_version 41050 (0.0009) [2023-10-12 21:42:19,861][44958] Updated weights for policy 0, policy_version 40840 (0.0009) [2023-10-12 21:42:20,219][44958] Updated weights for policy 0, policy_version 40850 (0.0010) [2023-10-12 21:42:20,597][44958] Updated weights for policy 0, policy_version 40860 (0.0008) [2023-10-12 21:42:21,058][44959] Updated weights for policy 1, policy_version 41060 (0.0010) [2023-10-12 21:42:21,430][44959] Updated weights for policy 1, policy_version 41070 (0.0011) [2023-10-12 21:42:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 83886080. Throughput: 0: 1648.2, 1: 1636.1. Samples: 20977074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:42:21,443][43579] Avg episode reward: [(0, '282.170'), (1, '275.180')] [2023-10-12 21:42:21,809][44959] Updated weights for policy 1, policy_version 41080 (0.0007) [2023-10-12 21:42:25,013][44958] Updated weights for policy 0, policy_version 40870 (0.0008) [2023-10-12 21:42:25,377][44958] Updated weights for policy 0, policy_version 40880 (0.0008) [2023-10-12 21:42:25,751][44958] Updated weights for policy 0, policy_version 40890 (0.0009) [2023-10-12 21:42:25,940][44959] Updated weights for policy 1, policy_version 41090 (0.0007) [2023-10-12 21:42:26,305][44959] Updated weights for policy 1, policy_version 41100 (0.0010) [2023-10-12 21:42:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 83951616. Throughput: 0: 1640.7, 1: 1639.6. Samples: 20996694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:42:26,443][43579] Avg episode reward: [(0, '281.790'), (1, '273.960')] [2023-10-12 21:42:26,677][44959] Updated weights for policy 1, policy_version 41110 (0.0010) [2023-10-12 21:42:27,039][44959] Updated weights for policy 1, policy_version 41120 (0.0008) [2023-10-12 21:42:30,205][44958] Updated weights for policy 0, policy_version 40900 (0.0008) [2023-10-12 21:42:30,579][44958] Updated weights for policy 0, policy_version 40910 (0.0007) [2023-10-12 21:42:30,952][44958] Updated weights for policy 0, policy_version 40920 (0.0009) [2023-10-12 21:42:31,173][44959] Updated weights for policy 1, policy_version 41130 (0.0008) [2023-10-12 21:42:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 84017152. Throughput: 0: 1638.8, 1: 1637.9. Samples: 21015986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:42:31,444][43579] Avg episode reward: [(0, '280.870'), (1, '274.980')] [2023-10-12 21:42:31,540][44959] Updated weights for policy 1, policy_version 41140 (0.0007) [2023-10-12 21:42:31,907][44959] Updated weights for policy 1, policy_version 41150 (0.0009) [2023-10-12 21:42:34,907][44958] Updated weights for policy 0, policy_version 40930 (0.0009) [2023-10-12 21:42:35,279][44958] Updated weights for policy 0, policy_version 40940 (0.0008) [2023-10-12 21:42:35,659][44958] Updated weights for policy 0, policy_version 40950 (0.0008) [2023-10-12 21:42:36,010][44959] Updated weights for policy 1, policy_version 41160 (0.0008) [2023-10-12 21:42:36,028][44958] Updated weights for policy 0, policy_version 40960 (0.0007) [2023-10-12 21:42:36,375][44959] Updated weights for policy 1, policy_version 41170 (0.0009) [2023-10-12 21:42:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84082688. Throughput: 0: 1635.5, 1: 1644.2. Samples: 21026130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:42:36,444][43579] Avg episode reward: [(0, '283.110'), (1, '273.030')] [2023-10-12 21:42:36,743][44959] Updated weights for policy 1, policy_version 41180 (0.0008) [2023-10-12 21:42:40,370][44958] Updated weights for policy 0, policy_version 40970 (0.0008) [2023-10-12 21:42:40,744][44958] Updated weights for policy 0, policy_version 40980 (0.0007) [2023-10-12 21:42:40,948][44959] Updated weights for policy 1, policy_version 41190 (0.0009) [2023-10-12 21:42:41,119][44958] Updated weights for policy 0, policy_version 40990 (0.0007) [2023-10-12 21:42:41,326][44959] Updated weights for policy 1, policy_version 41200 (0.0009) [2023-10-12 21:42:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84148224. Throughput: 0: 1639.6, 1: 1649.5. Samples: 21046492. Policy #0 lag: (min: 3.0, avg: 3.2, max: 11.0) [2023-10-12 21:42:41,444][43579] Avg episode reward: [(0, '277.380'), (1, '277.710')] [2023-10-12 21:42:41,692][44959] Updated weights for policy 1, policy_version 41210 (0.0008) [2023-10-12 21:42:45,185][44958] Updated weights for policy 0, policy_version 41000 (0.0007) [2023-10-12 21:42:45,551][44958] Updated weights for policy 0, policy_version 41010 (0.0007) [2023-10-12 21:42:45,659][44959] Updated weights for policy 1, policy_version 41220 (0.0008) [2023-10-12 21:42:45,911][44958] Updated weights for policy 0, policy_version 41020 (0.0007) [2023-10-12 21:42:46,025][44959] Updated weights for policy 1, policy_version 41230 (0.0009) [2023-10-12 21:42:46,394][44959] Updated weights for policy 1, policy_version 41240 (0.0009) [2023-10-12 21:42:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84213760. Throughput: 0: 1640.2, 1: 1647.6. Samples: 21065528. Policy #0 lag: (min: 3.0, avg: 3.2, max: 11.0) [2023-10-12 21:42:46,444][43579] Avg episode reward: [(0, '273.720'), (1, '283.960')] [2023-10-12 21:42:50,078][44958] Updated weights for policy 0, policy_version 41030 (0.0008) [2023-10-12 21:42:50,450][44958] Updated weights for policy 0, policy_version 41040 (0.0007) [2023-10-12 21:42:50,558][44959] Updated weights for policy 1, policy_version 41250 (0.0007) [2023-10-12 21:42:50,831][44958] Updated weights for policy 0, policy_version 41050 (0.0008) [2023-10-12 21:42:50,917][44959] Updated weights for policy 1, policy_version 41260 (0.0008) [2023-10-12 21:42:51,295][44959] Updated weights for policy 1, policy_version 41270 (0.0009) [2023-10-12 21:42:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84279296. Throughput: 0: 1638.6, 1: 1659.2. Samples: 21076084. Policy #0 lag: (min: 3.0, avg: 3.2, max: 11.0) [2023-10-12 21:42:51,444][43579] Avg episode reward: [(0, '267.850'), (1, '284.000')] [2023-10-12 21:42:51,670][44959] Updated weights for policy 1, policy_version 41280 (0.0010) [2023-10-12 21:42:54,888][44958] Updated weights for policy 0, policy_version 41060 (0.0008) [2023-10-12 21:42:55,257][44958] Updated weights for policy 0, policy_version 41070 (0.0007) [2023-10-12 21:42:55,625][44958] Updated weights for policy 0, policy_version 41080 (0.0008) [2023-10-12 21:42:55,967][44959] Updated weights for policy 1, policy_version 41290 (0.0007) [2023-10-12 21:42:56,337][44959] Updated weights for policy 1, policy_version 41300 (0.0008) [2023-10-12 21:42:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84344832. Throughput: 0: 1644.4, 1: 1657.1. Samples: 21095936. Policy #0 lag: (min: 3.0, avg: 3.2, max: 11.0) [2023-10-12 21:42:56,443][43579] Avg episode reward: [(0, '270.030'), (1, '283.560')] [2023-10-12 21:42:56,702][44959] Updated weights for policy 1, policy_version 41310 (0.0011) [2023-10-12 21:42:59,670][44958] Updated weights for policy 0, policy_version 41090 (0.0008) [2023-10-12 21:43:00,043][44958] Updated weights for policy 0, policy_version 41100 (0.0010) [2023-10-12 21:43:00,419][44958] Updated weights for policy 0, policy_version 41110 (0.0009) [2023-10-12 21:43:00,784][44958] Updated weights for policy 0, policy_version 41120 (0.0007) [2023-10-12 21:43:00,851][44959] Updated weights for policy 1, policy_version 41320 (0.0010) [2023-10-12 21:43:01,231][44959] Updated weights for policy 1, policy_version 41330 (0.0009) [2023-10-12 21:43:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84410368. Throughput: 0: 1644.8, 1: 1648.5. Samples: 21115092. Policy #0 lag: (min: 3.0, avg: 3.2, max: 11.0) [2023-10-12 21:43:01,443][43579] Avg episode reward: [(0, '263.630'), (1, '278.900')] [2023-10-12 21:43:01,596][44959] Updated weights for policy 1, policy_version 41340 (0.0009) [2023-10-12 21:43:05,048][44958] Updated weights for policy 0, policy_version 41130 (0.0008) [2023-10-12 21:43:05,419][44958] Updated weights for policy 0, policy_version 41140 (0.0008) [2023-10-12 21:43:05,782][44958] Updated weights for policy 0, policy_version 41150 (0.0010) [2023-10-12 21:43:05,790][44959] Updated weights for policy 1, policy_version 41350 (0.0009) [2023-10-12 21:43:06,164][44959] Updated weights for policy 1, policy_version 41360 (0.0007) [2023-10-12 21:43:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84475904. Throughput: 0: 1640.4, 1: 1657.1. Samples: 21125462. Policy #0 lag: (min: 3.0, avg: 3.2, max: 11.0) [2023-10-12 21:43:06,444][43579] Avg episode reward: [(0, '262.810'), (1, '279.870')] [2023-10-12 21:43:06,525][44959] Updated weights for policy 1, policy_version 41370 (0.0008) [2023-10-12 21:43:09,763][44958] Updated weights for policy 0, policy_version 41160 (0.0009) [2023-10-12 21:43:10,141][44958] Updated weights for policy 0, policy_version 41170 (0.0007) [2023-10-12 21:43:10,513][44958] Updated weights for policy 0, policy_version 41180 (0.0008) [2023-10-12 21:43:10,693][44959] Updated weights for policy 1, policy_version 41380 (0.0009) [2023-10-12 21:43:11,070][44959] Updated weights for policy 1, policy_version 41390 (0.0009) [2023-10-12 21:43:11,441][44959] Updated weights for policy 1, policy_version 41400 (0.0007) [2023-10-12 21:43:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84541440. Throughput: 0: 1639.2, 1: 1655.0. Samples: 21144932. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:43:11,443][43579] Avg episode reward: [(0, '263.730'), (1, '281.440')] [2023-10-12 21:43:14,790][44958] Updated weights for policy 0, policy_version 41190 (0.0008) [2023-10-12 21:43:15,166][44958] Updated weights for policy 0, policy_version 41200 (0.0009) [2023-10-12 21:43:15,460][44959] Updated weights for policy 1, policy_version 41410 (0.0008) [2023-10-12 21:43:15,532][44958] Updated weights for policy 0, policy_version 41210 (0.0008) [2023-10-12 21:43:15,826][44959] Updated weights for policy 1, policy_version 41420 (0.0007) [2023-10-12 21:43:16,196][44959] Updated weights for policy 1, policy_version 41430 (0.0007) [2023-10-12 21:43:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 84606976. Throughput: 0: 1641.3, 1: 1650.8. Samples: 21164126. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:43:16,443][43579] Avg episode reward: [(0, '261.060'), (1, '278.180')] [2023-10-12 21:43:16,566][44959] Updated weights for policy 1, policy_version 41440 (0.0008) [2023-10-12 21:43:19,618][44958] Updated weights for policy 0, policy_version 41220 (0.0009) [2023-10-12 21:43:19,994][44958] Updated weights for policy 0, policy_version 41230 (0.0007) [2023-10-12 21:43:20,357][44958] Updated weights for policy 0, policy_version 41240 (0.0011) [2023-10-12 21:43:20,933][44959] Updated weights for policy 1, policy_version 41450 (0.0007) [2023-10-12 21:43:21,306][44959] Updated weights for policy 1, policy_version 41460 (0.0009) [2023-10-12 21:43:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 84672512. Throughput: 0: 1647.1, 1: 1657.0. Samples: 21174814. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:43:21,444][43579] Avg episode reward: [(0, '262.210'), (1, '280.540')] [2023-10-12 21:43:21,681][44959] Updated weights for policy 1, policy_version 41470 (0.0009) [2023-10-12 21:43:24,652][44958] Updated weights for policy 0, policy_version 41250 (0.0011) [2023-10-12 21:43:25,018][44958] Updated weights for policy 0, policy_version 41260 (0.0010) [2023-10-12 21:43:25,400][44958] Updated weights for policy 0, policy_version 41270 (0.0008) [2023-10-12 21:43:25,560][44959] Updated weights for policy 1, policy_version 41480 (0.0008) [2023-10-12 21:43:25,780][44958] Updated weights for policy 0, policy_version 41280 (0.0009) [2023-10-12 21:43:25,931][44959] Updated weights for policy 1, policy_version 41490 (0.0007) [2023-10-12 21:43:26,305][44959] Updated weights for policy 1, policy_version 41500 (0.0007) [2023-10-12 21:43:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84738048. Throughput: 0: 1633.6, 1: 1653.0. Samples: 21194388. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:43:26,443][43579] Avg episode reward: [(0, '261.700'), (1, '278.800')] [2023-10-12 21:43:29,956][44958] Updated weights for policy 0, policy_version 41290 (0.0010) [2023-10-12 21:43:30,325][44958] Updated weights for policy 0, policy_version 41300 (0.0008) [2023-10-12 21:43:30,388][44959] Updated weights for policy 1, policy_version 41510 (0.0008) [2023-10-12 21:43:30,692][44958] Updated weights for policy 0, policy_version 41310 (0.0009) [2023-10-12 21:43:30,759][44959] Updated weights for policy 1, policy_version 41520 (0.0007) [2023-10-12 21:43:31,121][44959] Updated weights for policy 1, policy_version 41530 (0.0008) [2023-10-12 21:43:31,442][43579] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 84836352. Throughput: 0: 1636.1, 1: 1647.0. Samples: 21213268. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:43:31,443][43579] Avg episode reward: [(0, '265.820'), (1, '279.640')] [2023-10-12 21:43:34,775][44958] Updated weights for policy 0, policy_version 41320 (0.0010) [2023-10-12 21:43:35,151][44958] Updated weights for policy 0, policy_version 41330 (0.0011) [2023-10-12 21:43:35,520][44958] Updated weights for policy 0, policy_version 41340 (0.0009) [2023-10-12 21:43:35,535][44959] Updated weights for policy 1, policy_version 41540 (0.0008) [2023-10-12 21:43:35,897][44959] Updated weights for policy 1, policy_version 41550 (0.0010) [2023-10-12 21:43:36,269][44959] Updated weights for policy 1, policy_version 41560 (0.0008) [2023-10-12 21:43:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 84869120. Throughput: 0: 1643.1, 1: 1649.9. Samples: 21224266. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 21:43:36,444][43579] Avg episode reward: [(0, '266.160'), (1, '280.140')] [2023-10-12 21:43:39,797][44958] Updated weights for policy 0, policy_version 41350 (0.0009) [2023-10-12 21:43:40,168][44958] Updated weights for policy 0, policy_version 41360 (0.0011) [2023-10-12 21:43:40,297][44959] Updated weights for policy 1, policy_version 41570 (0.0009) [2023-10-12 21:43:40,535][44958] Updated weights for policy 0, policy_version 41370 (0.0009) [2023-10-12 21:43:40,678][44959] Updated weights for policy 1, policy_version 41580 (0.0007) [2023-10-12 21:43:41,052][44959] Updated weights for policy 1, policy_version 41590 (0.0009) [2023-10-12 21:43:41,422][44959] Updated weights for policy 1, policy_version 41600 (0.0007) [2023-10-12 21:43:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 84967424. Throughput: 0: 1634.2, 1: 1652.8. Samples: 21243852. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:43:41,443][43579] Avg episode reward: [(0, '270.010'), (1, '277.640')] [2023-10-12 21:43:44,663][44958] Updated weights for policy 0, policy_version 41380 (0.0007) [2023-10-12 21:43:45,030][44958] Updated weights for policy 0, policy_version 41390 (0.0007) [2023-10-12 21:43:45,406][44958] Updated weights for policy 0, policy_version 41400 (0.0008) [2023-10-12 21:43:45,599][44959] Updated weights for policy 1, policy_version 41610 (0.0008) [2023-10-12 21:43:45,965][44959] Updated weights for policy 1, policy_version 41620 (0.0008) [2023-10-12 21:43:46,334][44959] Updated weights for policy 1, policy_version 41630 (0.0009) [2023-10-12 21:43:46,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 85032960. Throughput: 0: 1635.7, 1: 1642.7. Samples: 21262620. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:43:46,444][43579] Avg episode reward: [(0, '271.190'), (1, '279.650')] [2023-10-12 21:43:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000041632_42631168.pth... [2023-10-12 21:43:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000041408_42401792.pth... [2023-10-12 21:43:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000039872_40828928.pth [2023-10-12 21:43:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000040064_41025536.pth [2023-10-12 21:43:49,462][44958] Updated weights for policy 0, policy_version 41410 (0.0007) [2023-10-12 21:43:49,837][44958] Updated weights for policy 0, policy_version 41420 (0.0007) [2023-10-12 21:43:50,206][44958] Updated weights for policy 0, policy_version 41430 (0.0007) [2023-10-12 21:43:50,577][44958] Updated weights for policy 0, policy_version 41440 (0.0009) [2023-10-12 21:43:50,640][44959] Updated weights for policy 1, policy_version 41640 (0.0009) [2023-10-12 21:43:51,009][44959] Updated weights for policy 1, policy_version 41650 (0.0008) [2023-10-12 21:43:51,367][44959] Updated weights for policy 1, policy_version 41660 (0.0009) [2023-10-12 21:43:51,442][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85065728. Throughput: 0: 1640.1, 1: 1649.1. Samples: 21273474. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:43:51,443][43579] Avg episode reward: [(0, '275.120'), (1, '279.130')] [2023-10-12 21:43:54,682][44958] Updated weights for policy 0, policy_version 41450 (0.0007) [2023-10-12 21:43:55,064][44958] Updated weights for policy 0, policy_version 41460 (0.0008) [2023-10-12 21:43:55,424][44958] Updated weights for policy 0, policy_version 41470 (0.0009) [2023-10-12 21:43:55,555][44959] Updated weights for policy 1, policy_version 41670 (0.0009) [2023-10-12 21:43:55,918][44959] Updated weights for policy 1, policy_version 41680 (0.0007) [2023-10-12 21:43:56,287][44959] Updated weights for policy 1, policy_version 41690 (0.0008) [2023-10-12 21:43:56,443][43579] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85131264. Throughput: 0: 1637.1, 1: 1653.9. Samples: 21293026. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:43:56,444][43579] Avg episode reward: [(0, '273.660'), (1, '278.140')] [2023-10-12 21:43:59,535][44958] Updated weights for policy 0, policy_version 41480 (0.0010) [2023-10-12 21:43:59,909][44958] Updated weights for policy 0, policy_version 41490 (0.0009) [2023-10-12 21:44:00,285][44958] Updated weights for policy 0, policy_version 41500 (0.0008) [2023-10-12 21:44:00,480][44959] Updated weights for policy 1, policy_version 41700 (0.0010) [2023-10-12 21:44:00,851][44959] Updated weights for policy 1, policy_version 41710 (0.0007) [2023-10-12 21:44:01,208][44959] Updated weights for policy 1, policy_version 41720 (0.0007) [2023-10-12 21:44:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 85196800. Throughput: 0: 1648.1, 1: 1645.6. Samples: 21312340. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:44:01,443][43579] Avg episode reward: [(0, '273.130'), (1, '278.000')] [2023-10-12 21:44:04,441][44958] Updated weights for policy 0, policy_version 41510 (0.0008) [2023-10-12 21:44:04,812][44958] Updated weights for policy 0, policy_version 41520 (0.0007) [2023-10-12 21:44:05,187][44958] Updated weights for policy 0, policy_version 41530 (0.0009) [2023-10-12 21:44:05,423][44959] Updated weights for policy 1, policy_version 41730 (0.0008) [2023-10-12 21:44:05,791][44959] Updated weights for policy 1, policy_version 41740 (0.0009) [2023-10-12 21:44:06,160][44959] Updated weights for policy 1, policy_version 41750 (0.0009) [2023-10-12 21:44:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 85262336. Throughput: 0: 1649.6, 1: 1648.9. Samples: 21323248. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 21:44:06,443][43579] Avg episode reward: [(0, '276.870'), (1, '274.990')] [2023-10-12 21:44:06,530][44959] Updated weights for policy 1, policy_version 41760 (0.0008) [2023-10-12 21:44:09,391][44958] Updated weights for policy 0, policy_version 41540 (0.0009) [2023-10-12 21:44:09,759][44958] Updated weights for policy 0, policy_version 41550 (0.0011) [2023-10-12 21:44:10,136][44958] Updated weights for policy 0, policy_version 41560 (0.0009) [2023-10-12 21:44:10,597][44959] Updated weights for policy 1, policy_version 41770 (0.0008) [2023-10-12 21:44:10,975][44959] Updated weights for policy 1, policy_version 41780 (0.0010) [2023-10-12 21:44:11,348][44959] Updated weights for policy 1, policy_version 41790 (0.0008) [2023-10-12 21:44:11,442][43579] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 85360640. Throughput: 0: 1644.1, 1: 1649.4. Samples: 21342598. Policy #0 lag: (min: 8.0, avg: 26.9, max: 40.0) [2023-10-12 21:44:11,443][43579] Avg episode reward: [(0, '279.140'), (1, '273.590')] [2023-10-12 21:44:14,508][44958] Updated weights for policy 0, policy_version 41570 (0.0007) [2023-10-12 21:44:14,876][44958] Updated weights for policy 0, policy_version 41580 (0.0009) [2023-10-12 21:44:15,240][44958] Updated weights for policy 0, policy_version 41590 (0.0009) [2023-10-12 21:44:15,608][44958] Updated weights for policy 0, policy_version 41600 (0.0009) [2023-10-12 21:44:15,612][44959] Updated weights for policy 1, policy_version 41800 (0.0008) [2023-10-12 21:44:15,979][44959] Updated weights for policy 1, policy_version 41810 (0.0007) [2023-10-12 21:44:16,350][44959] Updated weights for policy 1, policy_version 41820 (0.0009) [2023-10-12 21:44:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 85393408. Throughput: 0: 1653.6, 1: 1640.6. Samples: 21361508. Policy #0 lag: (min: 8.0, avg: 26.9, max: 40.0) [2023-10-12 21:44:16,443][43579] Avg episode reward: [(0, '278.540'), (1, '273.030')] [2023-10-12 21:44:19,835][44958] Updated weights for policy 0, policy_version 41610 (0.0010) [2023-10-12 21:44:20,214][44958] Updated weights for policy 0, policy_version 41620 (0.0009) [2023-10-12 21:44:20,340][44959] Updated weights for policy 1, policy_version 41830 (0.0007) [2023-10-12 21:44:20,590][44958] Updated weights for policy 0, policy_version 41630 (0.0009) [2023-10-12 21:44:20,706][44959] Updated weights for policy 1, policy_version 41840 (0.0009) [2023-10-12 21:44:21,072][44959] Updated weights for policy 1, policy_version 41850 (0.0010) [2023-10-12 21:44:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 85491712. Throughput: 0: 1647.7, 1: 1644.8. Samples: 21372430. Policy #0 lag: (min: 8.0, avg: 26.9, max: 40.0) [2023-10-12 21:44:21,444][43579] Avg episode reward: [(0, '279.460'), (1, '274.350')] [2023-10-12 21:44:24,779][44958] Updated weights for policy 0, policy_version 41640 (0.0008) [2023-10-12 21:44:25,160][44958] Updated weights for policy 0, policy_version 41650 (0.0009) [2023-10-12 21:44:25,313][44959] Updated weights for policy 1, policy_version 41860 (0.0010) [2023-10-12 21:44:25,521][44958] Updated weights for policy 0, policy_version 41660 (0.0008) [2023-10-12 21:44:25,697][44959] Updated weights for policy 1, policy_version 41870 (0.0007) [2023-10-12 21:44:26,070][44959] Updated weights for policy 1, policy_version 41880 (0.0007) [2023-10-12 21:44:26,442][43579] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 85557248. Throughput: 0: 1646.8, 1: 1640.9. Samples: 21391800. Policy #0 lag: (min: 8.0, avg: 26.9, max: 40.0) [2023-10-12 21:44:26,443][43579] Avg episode reward: [(0, '280.750'), (1, '276.050')] [2023-10-12 21:44:29,673][44958] Updated weights for policy 0, policy_version 41670 (0.0009) [2023-10-12 21:44:30,052][44958] Updated weights for policy 0, policy_version 41680 (0.0010) [2023-10-12 21:44:30,246][44959] Updated weights for policy 1, policy_version 41890 (0.0008) [2023-10-12 21:44:30,418][44958] Updated weights for policy 0, policy_version 41690 (0.0007) [2023-10-12 21:44:30,607][44959] Updated weights for policy 1, policy_version 41900 (0.0007) [2023-10-12 21:44:30,969][44959] Updated weights for policy 1, policy_version 41910 (0.0011) [2023-10-12 21:44:31,335][44959] Updated weights for policy 1, policy_version 41920 (0.0011) [2023-10-12 21:44:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85622784. Throughput: 0: 1648.8, 1: 1638.6. Samples: 21410550. Policy #0 lag: (min: 8.0, avg: 26.9, max: 40.0) [2023-10-12 21:44:31,444][43579] Avg episode reward: [(0, '280.340'), (1, '278.590')] [2023-10-12 21:44:34,545][44958] Updated weights for policy 0, policy_version 41700 (0.0008) [2023-10-12 21:44:34,909][44958] Updated weights for policy 0, policy_version 41710 (0.0010) [2023-10-12 21:44:35,278][44958] Updated weights for policy 0, policy_version 41720 (0.0008) [2023-10-12 21:44:35,540][44959] Updated weights for policy 1, policy_version 41930 (0.0008) [2023-10-12 21:44:35,912][44959] Updated weights for policy 1, policy_version 41940 (0.0010) [2023-10-12 21:44:36,289][44959] Updated weights for policy 1, policy_version 41950 (0.0010) [2023-10-12 21:44:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 85688320. Throughput: 0: 1647.9, 1: 1645.7. Samples: 21421684. Policy #0 lag: (min: 8.0, avg: 26.9, max: 40.0) [2023-10-12 21:44:36,443][43579] Avg episode reward: [(0, '274.070'), (1, '283.880')] [2023-10-12 21:44:39,584][44958] Updated weights for policy 0, policy_version 41730 (0.0008) [2023-10-12 21:44:39,954][44958] Updated weights for policy 0, policy_version 41740 (0.0007) [2023-10-12 21:44:40,339][44958] Updated weights for policy 0, policy_version 41750 (0.0008) [2023-10-12 21:44:40,365][44959] Updated weights for policy 1, policy_version 41960 (0.0008) [2023-10-12 21:44:40,698][44958] Updated weights for policy 0, policy_version 41760 (0.0008) [2023-10-12 21:44:40,731][44959] Updated weights for policy 1, policy_version 41970 (0.0009) [2023-10-12 21:44:41,103][44959] Updated weights for policy 1, policy_version 41980 (0.0009) [2023-10-12 21:44:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 85753856. Throughput: 0: 1642.8, 1: 1646.4. Samples: 21441038. Policy #0 lag: (min: 25.0, avg: 28.3, max: 57.0) [2023-10-12 21:44:41,444][43579] Avg episode reward: [(0, '275.800'), (1, '282.950')] [2023-10-12 21:44:44,870][44958] Updated weights for policy 0, policy_version 41770 (0.0011) [2023-10-12 21:44:45,236][44958] Updated weights for policy 0, policy_version 41780 (0.0009) [2023-10-12 21:44:45,446][44959] Updated weights for policy 1, policy_version 41990 (0.0008) [2023-10-12 21:44:45,606][44958] Updated weights for policy 0, policy_version 41790 (0.0007) [2023-10-12 21:44:45,801][44959] Updated weights for policy 1, policy_version 42000 (0.0010) [2023-10-12 21:44:46,172][44959] Updated weights for policy 1, policy_version 42010 (0.0008) [2023-10-12 21:44:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 85819392. Throughput: 0: 1633.2, 1: 1645.5. Samples: 21459878. Policy #0 lag: (min: 25.0, avg: 28.3, max: 57.0) [2023-10-12 21:44:46,443][43579] Avg episode reward: [(0, '272.200'), (1, '283.400')] [2023-10-12 21:44:49,883][44958] Updated weights for policy 0, policy_version 41800 (0.0009) [2023-10-12 21:44:50,180][44959] Updated weights for policy 1, policy_version 42020 (0.0008) [2023-10-12 21:44:50,251][44958] Updated weights for policy 0, policy_version 41810 (0.0007) [2023-10-12 21:44:50,553][44959] Updated weights for policy 1, policy_version 42030 (0.0007) [2023-10-12 21:44:50,623][44958] Updated weights for policy 0, policy_version 41820 (0.0008) [2023-10-12 21:44:50,922][44959] Updated weights for policy 1, policy_version 42040 (0.0008) [2023-10-12 21:44:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 85884928. Throughput: 0: 1627.2, 1: 1649.7. Samples: 21470710. Policy #0 lag: (min: 25.0, avg: 28.3, max: 57.0) [2023-10-12 21:44:51,444][43579] Avg episode reward: [(0, '266.560'), (1, '284.510')] [2023-10-12 21:44:54,809][44958] Updated weights for policy 0, policy_version 41830 (0.0008) [2023-10-12 21:44:55,139][44959] Updated weights for policy 1, policy_version 42050 (0.0009) [2023-10-12 21:44:55,174][44958] Updated weights for policy 0, policy_version 41840 (0.0007) [2023-10-12 21:44:55,507][44959] Updated weights for policy 1, policy_version 42060 (0.0008) [2023-10-12 21:44:55,551][44958] Updated weights for policy 0, policy_version 41850 (0.0007) [2023-10-12 21:44:55,874][44959] Updated weights for policy 1, policy_version 42070 (0.0010) [2023-10-12 21:44:56,243][44959] Updated weights for policy 1, policy_version 42080 (0.0010) [2023-10-12 21:44:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 85950464. Throughput: 0: 1637.1, 1: 1643.6. Samples: 21490230. Policy #0 lag: (min: 25.0, avg: 28.3, max: 57.0) [2023-10-12 21:44:56,443][43579] Avg episode reward: [(0, '269.100'), (1, '281.940')] [2023-10-12 21:44:59,532][44958] Updated weights for policy 0, policy_version 41860 (0.0007) [2023-10-12 21:44:59,893][44958] Updated weights for policy 0, policy_version 41870 (0.0008) [2023-10-12 21:45:00,275][44958] Updated weights for policy 0, policy_version 41880 (0.0009) [2023-10-12 21:45:00,377][44959] Updated weights for policy 1, policy_version 42090 (0.0008) [2023-10-12 21:45:00,747][44959] Updated weights for policy 1, policy_version 42100 (0.0009) [2023-10-12 21:45:01,116][44959] Updated weights for policy 1, policy_version 42110 (0.0010) [2023-10-12 21:45:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 86016000. Throughput: 0: 1635.9, 1: 1638.9. Samples: 21508874. Policy #0 lag: (min: 25.0, avg: 28.3, max: 57.0) [2023-10-12 21:45:01,443][43579] Avg episode reward: [(0, '261.650'), (1, '281.810')] [2023-10-12 21:45:04,676][44958] Updated weights for policy 0, policy_version 41890 (0.0008) [2023-10-12 21:45:05,086][44958] Updated weights for policy 0, policy_version 41900 (0.0008) [2023-10-12 21:45:05,378][44959] Updated weights for policy 1, policy_version 42120 (0.0007) [2023-10-12 21:45:05,448][44958] Updated weights for policy 0, policy_version 41910 (0.0009) [2023-10-12 21:45:05,749][44959] Updated weights for policy 1, policy_version 42130 (0.0007) [2023-10-12 21:45:05,817][44958] Updated weights for policy 0, policy_version 41920 (0.0008) [2023-10-12 21:45:06,111][44959] Updated weights for policy 1, policy_version 42140 (0.0010) [2023-10-12 21:45:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 86081536. Throughput: 0: 1633.0, 1: 1639.8. Samples: 21519706. Policy #0 lag: (min: 25.0, avg: 28.3, max: 57.0) [2023-10-12 21:45:06,443][43579] Avg episode reward: [(0, '265.720'), (1, '284.580')] [2023-10-12 21:45:09,834][44958] Updated weights for policy 0, policy_version 41930 (0.0010) [2023-10-12 21:45:10,207][44958] Updated weights for policy 0, policy_version 41940 (0.0007) [2023-10-12 21:45:10,400][44959] Updated weights for policy 1, policy_version 42150 (0.0009) [2023-10-12 21:45:10,571][44958] Updated weights for policy 0, policy_version 41950 (0.0007) [2023-10-12 21:45:10,778][44959] Updated weights for policy 1, policy_version 42160 (0.0009) [2023-10-12 21:45:11,149][44959] Updated weights for policy 1, policy_version 42170 (0.0008) [2023-10-12 21:45:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86147072. Throughput: 0: 1633.6, 1: 1640.5. Samples: 21539138. Policy #0 lag: (min: 9.0, avg: 19.8, max: 41.0) [2023-10-12 21:45:11,443][43579] Avg episode reward: [(0, '263.760'), (1, '278.830')] [2023-10-12 21:45:14,625][44958] Updated weights for policy 0, policy_version 41960 (0.0009) [2023-10-12 21:45:15,000][44958] Updated weights for policy 0, policy_version 41970 (0.0010) [2023-10-12 21:45:15,287][44959] Updated weights for policy 1, policy_version 42180 (0.0007) [2023-10-12 21:45:15,378][44958] Updated weights for policy 0, policy_version 41980 (0.0010) [2023-10-12 21:45:15,657][44959] Updated weights for policy 1, policy_version 42190 (0.0009) [2023-10-12 21:45:16,020][44959] Updated weights for policy 1, policy_version 42200 (0.0009) [2023-10-12 21:45:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 86212608. Throughput: 0: 1633.2, 1: 1643.8. Samples: 21558012. Policy #0 lag: (min: 9.0, avg: 19.8, max: 41.0) [2023-10-12 21:45:16,443][43579] Avg episode reward: [(0, '266.890'), (1, '276.970')] [2023-10-12 21:45:19,702][44958] Updated weights for policy 0, policy_version 41990 (0.0009) [2023-10-12 21:45:20,080][44958] Updated weights for policy 0, policy_version 42000 (0.0007) [2023-10-12 21:45:20,177][44959] Updated weights for policy 1, policy_version 42210 (0.0008) [2023-10-12 21:45:20,463][44958] Updated weights for policy 0, policy_version 42010 (0.0008) [2023-10-12 21:45:20,539][44959] Updated weights for policy 1, policy_version 42220 (0.0008) [2023-10-12 21:45:20,903][44959] Updated weights for policy 1, policy_version 42230 (0.0010) [2023-10-12 21:45:21,274][44959] Updated weights for policy 1, policy_version 42240 (0.0011) [2023-10-12 21:45:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86278144. Throughput: 0: 1631.3, 1: 1637.7. Samples: 21568788. Policy #0 lag: (min: 9.0, avg: 19.8, max: 41.0) [2023-10-12 21:45:21,443][43579] Avg episode reward: [(0, '269.490'), (1, '277.400')] [2023-10-12 21:45:24,624][44958] Updated weights for policy 0, policy_version 42020 (0.0009) [2023-10-12 21:45:25,000][44958] Updated weights for policy 0, policy_version 42030 (0.0009) [2023-10-12 21:45:25,375][44958] Updated weights for policy 0, policy_version 42040 (0.0009) [2023-10-12 21:45:25,577][44959] Updated weights for policy 1, policy_version 42250 (0.0008) [2023-10-12 21:45:25,956][44959] Updated weights for policy 1, policy_version 42260 (0.0009) [2023-10-12 21:45:26,324][44959] Updated weights for policy 1, policy_version 42270 (0.0008) [2023-10-12 21:45:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86343680. Throughput: 0: 1637.2, 1: 1634.3. Samples: 21588258. Policy #0 lag: (min: 9.0, avg: 19.8, max: 41.0) [2023-10-12 21:45:26,443][43579] Avg episode reward: [(0, '271.920'), (1, '274.670')] [2023-10-12 21:45:29,611][44958] Updated weights for policy 0, policy_version 42050 (0.0009) [2023-10-12 21:45:29,986][44958] Updated weights for policy 0, policy_version 42060 (0.0008) [2023-10-12 21:45:30,344][44958] Updated weights for policy 0, policy_version 42070 (0.0008) [2023-10-12 21:45:30,359][44959] Updated weights for policy 1, policy_version 42280 (0.0009) [2023-10-12 21:45:30,713][44958] Updated weights for policy 0, policy_version 42080 (0.0009) [2023-10-12 21:45:30,721][44959] Updated weights for policy 1, policy_version 42290 (0.0009) [2023-10-12 21:45:31,085][44959] Updated weights for policy 1, policy_version 42300 (0.0010) [2023-10-12 21:45:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86409216. Throughput: 0: 1637.8, 1: 1629.2. Samples: 21606894. Policy #0 lag: (min: 9.0, avg: 19.8, max: 41.0) [2023-10-12 21:45:31,443][43579] Avg episode reward: [(0, '274.770'), (1, '275.400')] [2023-10-12 21:45:34,966][44958] Updated weights for policy 0, policy_version 42090 (0.0009) [2023-10-12 21:45:35,342][44958] Updated weights for policy 0, policy_version 42100 (0.0010) [2023-10-12 21:45:35,369][44959] Updated weights for policy 1, policy_version 42310 (0.0011) [2023-10-12 21:45:35,704][44958] Updated weights for policy 0, policy_version 42110 (0.0009) [2023-10-12 21:45:35,742][44959] Updated weights for policy 1, policy_version 42320 (0.0009) [2023-10-12 21:45:36,115][44959] Updated weights for policy 1, policy_version 42330 (0.0007) [2023-10-12 21:45:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86474752. Throughput: 0: 1642.6, 1: 1628.4. Samples: 21617906. Policy #0 lag: (min: 9.0, avg: 19.8, max: 41.0) [2023-10-12 21:45:36,444][43579] Avg episode reward: [(0, '278.060'), (1, '278.090')] [2023-10-12 21:45:40,064][44958] Updated weights for policy 0, policy_version 42120 (0.0009) [2023-10-12 21:45:40,366][44959] Updated weights for policy 1, policy_version 42340 (0.0008) [2023-10-12 21:45:40,437][44958] Updated weights for policy 0, policy_version 42130 (0.0010) [2023-10-12 21:45:40,732][44959] Updated weights for policy 1, policy_version 42350 (0.0008) [2023-10-12 21:45:40,815][44958] Updated weights for policy 0, policy_version 42140 (0.0007) [2023-10-12 21:45:41,106][44959] Updated weights for policy 1, policy_version 42360 (0.0008) [2023-10-12 21:45:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86540288. Throughput: 0: 1640.7, 1: 1628.0. Samples: 21637322. Policy #0 lag: (min: 44.0, avg: 48.0, max: 48.0) [2023-10-12 21:45:41,444][43579] Avg episode reward: [(0, '280.640'), (1, '280.750')] [2023-10-12 21:45:44,877][44958] Updated weights for policy 0, policy_version 42150 (0.0008) [2023-10-12 21:45:45,246][44959] Updated weights for policy 1, policy_version 42370 (0.0007) [2023-10-12 21:45:45,248][44958] Updated weights for policy 0, policy_version 42160 (0.0008) [2023-10-12 21:45:45,617][44959] Updated weights for policy 1, policy_version 42380 (0.0009) [2023-10-12 21:45:45,619][44958] Updated weights for policy 0, policy_version 42170 (0.0008) [2023-10-12 21:45:45,986][44959] Updated weights for policy 1, policy_version 42390 (0.0008) [2023-10-12 21:45:46,358][44959] Updated weights for policy 1, policy_version 42400 (0.0009) [2023-10-12 21:45:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86605824. Throughput: 0: 1631.6, 1: 1635.9. Samples: 21655910. Policy #0 lag: (min: 44.0, avg: 48.0, max: 48.0) [2023-10-12 21:45:46,443][43579] Avg episode reward: [(0, '276.950'), (1, '282.600')] [2023-10-12 21:45:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000042176_43188224.pth... [2023-10-12 21:45:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000042400_43417600.pth... [2023-10-12 21:45:46,481][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000040640_41615360.pth [2023-10-12 21:45:46,490][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000040864_41844736.pth [2023-10-12 21:45:49,959][44958] Updated weights for policy 0, policy_version 42180 (0.0008) [2023-10-12 21:45:50,320][44958] Updated weights for policy 0, policy_version 42190 (0.0009) [2023-10-12 21:45:50,484][44959] Updated weights for policy 1, policy_version 42410 (0.0010) [2023-10-12 21:45:50,691][44958] Updated weights for policy 0, policy_version 42200 (0.0009) [2023-10-12 21:45:50,847][44959] Updated weights for policy 1, policy_version 42420 (0.0008) [2023-10-12 21:45:51,234][44959] Updated weights for policy 1, policy_version 42430 (0.0010) [2023-10-12 21:45:51,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86671360. Throughput: 0: 1633.6, 1: 1635.6. Samples: 21666824. Policy #0 lag: (min: 44.0, avg: 48.0, max: 48.0) [2023-10-12 21:45:51,443][43579] Avg episode reward: [(0, '274.090'), (1, '282.930')] [2023-10-12 21:45:55,007][44958] Updated weights for policy 0, policy_version 42210 (0.0008) [2023-10-12 21:45:55,357][44959] Updated weights for policy 1, policy_version 42440 (0.0009) [2023-10-12 21:45:55,375][44958] Updated weights for policy 0, policy_version 42220 (0.0008) [2023-10-12 21:45:55,722][44959] Updated weights for policy 1, policy_version 42450 (0.0009) [2023-10-12 21:45:55,754][44958] Updated weights for policy 0, policy_version 42230 (0.0008) [2023-10-12 21:45:56,092][44959] Updated weights for policy 1, policy_version 42460 (0.0007) [2023-10-12 21:45:56,134][44958] Updated weights for policy 0, policy_version 42240 (0.0009) [2023-10-12 21:45:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86736896. Throughput: 0: 1640.8, 1: 1635.0. Samples: 21686550. Policy #0 lag: (min: 44.0, avg: 48.0, max: 48.0) [2023-10-12 21:45:56,443][43579] Avg episode reward: [(0, '276.250'), (1, '289.530')] [2023-10-12 21:45:56,444][44583] Saving new best policy, reward=289.530! [2023-10-12 21:46:00,269][44958] Updated weights for policy 0, policy_version 42250 (0.0009) [2023-10-12 21:46:00,273][44959] Updated weights for policy 1, policy_version 42470 (0.0009) [2023-10-12 21:46:00,631][44958] Updated weights for policy 0, policy_version 42260 (0.0010) [2023-10-12 21:46:00,633][44959] Updated weights for policy 1, policy_version 42480 (0.0007) [2023-10-12 21:46:00,995][44958] Updated weights for policy 0, policy_version 42270 (0.0008) [2023-10-12 21:46:01,006][44959] Updated weights for policy 1, policy_version 42490 (0.0007) [2023-10-12 21:46:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86802432. Throughput: 0: 1630.3, 1: 1627.3. Samples: 21704602. Policy #0 lag: (min: 44.0, avg: 48.0, max: 48.0) [2023-10-12 21:46:01,443][43579] Avg episode reward: [(0, '274.520'), (1, '289.670')] [2023-10-12 21:46:01,451][44583] Saving new best policy, reward=289.670! [2023-10-12 21:46:05,118][44958] Updated weights for policy 0, policy_version 42280 (0.0008) [2023-10-12 21:46:05,214][44959] Updated weights for policy 1, policy_version 42500 (0.0007) [2023-10-12 21:46:05,493][44958] Updated weights for policy 0, policy_version 42290 (0.0009) [2023-10-12 21:46:05,582][44959] Updated weights for policy 1, policy_version 42510 (0.0007) [2023-10-12 21:46:05,872][44958] Updated weights for policy 0, policy_version 42300 (0.0008) [2023-10-12 21:46:05,951][44959] Updated weights for policy 1, policy_version 42520 (0.0009) [2023-10-12 21:46:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86867968. Throughput: 0: 1633.0, 1: 1638.8. Samples: 21716018. Policy #0 lag: (min: 44.0, avg: 48.0, max: 48.0) [2023-10-12 21:46:06,444][43579] Avg episode reward: [(0, '272.640'), (1, '288.660')] [2023-10-12 21:46:10,100][44958] Updated weights for policy 0, policy_version 42310 (0.0008) [2023-10-12 21:46:10,175][44959] Updated weights for policy 1, policy_version 42530 (0.0010) [2023-10-12 21:46:10,476][44958] Updated weights for policy 0, policy_version 42320 (0.0007) [2023-10-12 21:46:10,543][44959] Updated weights for policy 1, policy_version 42540 (0.0008) [2023-10-12 21:46:10,854][44958] Updated weights for policy 0, policy_version 42330 (0.0007) [2023-10-12 21:46:10,907][44959] Updated weights for policy 1, policy_version 42550 (0.0007) [2023-10-12 21:46:11,269][44959] Updated weights for policy 1, policy_version 42560 (0.0008) [2023-10-12 21:46:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 86933504. Throughput: 0: 1638.1, 1: 1638.8. Samples: 21735718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:11,444][43579] Avg episode reward: [(0, '274.130'), (1, '285.040')] [2023-10-12 21:46:14,847][44958] Updated weights for policy 0, policy_version 42340 (0.0009) [2023-10-12 21:46:15,221][44958] Updated weights for policy 0, policy_version 42350 (0.0011) [2023-10-12 21:46:15,587][44958] Updated weights for policy 0, policy_version 42360 (0.0009) [2023-10-12 21:46:15,690][44959] Updated weights for policy 1, policy_version 42570 (0.0009) [2023-10-12 21:46:16,056][44959] Updated weights for policy 1, policy_version 42580 (0.0009) [2023-10-12 21:46:16,419][44959] Updated weights for policy 1, policy_version 42590 (0.0007) [2023-10-12 21:46:16,443][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 86966272. Throughput: 0: 1626.5, 1: 1642.6. Samples: 21754004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:16,444][43579] Avg episode reward: [(0, '279.080'), (1, '284.760')] [2023-10-12 21:46:19,894][44958] Updated weights for policy 0, policy_version 42370 (0.0007) [2023-10-12 21:46:20,270][44958] Updated weights for policy 0, policy_version 42380 (0.0009) [2023-10-12 21:46:20,574][44959] Updated weights for policy 1, policy_version 42600 (0.0011) [2023-10-12 21:46:20,643][44958] Updated weights for policy 0, policy_version 42390 (0.0007) [2023-10-12 21:46:20,942][44959] Updated weights for policy 1, policy_version 42610 (0.0009) [2023-10-12 21:46:21,020][44958] Updated weights for policy 0, policy_version 42400 (0.0008) [2023-10-12 21:46:21,299][44959] Updated weights for policy 1, policy_version 42620 (0.0008) [2023-10-12 21:46:21,443][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 87031808. Throughput: 0: 1625.5, 1: 1640.9. Samples: 21764896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:21,444][43579] Avg episode reward: [(0, '279.980'), (1, '278.980')] [2023-10-12 21:46:25,274][44958] Updated weights for policy 0, policy_version 42410 (0.0009) [2023-10-12 21:46:25,388][44959] Updated weights for policy 1, policy_version 42630 (0.0008) [2023-10-12 21:46:25,651][44958] Updated weights for policy 0, policy_version 42420 (0.0008) [2023-10-12 21:46:25,754][44959] Updated weights for policy 1, policy_version 42640 (0.0007) [2023-10-12 21:46:26,029][44958] Updated weights for policy 0, policy_version 42430 (0.0008) [2023-10-12 21:46:26,114][44959] Updated weights for policy 1, policy_version 42650 (0.0011) [2023-10-12 21:46:26,442][43579] Fps is (10 sec: 16384.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87130112. Throughput: 0: 1634.4, 1: 1640.6. Samples: 21784698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:26,443][43579] Avg episode reward: [(0, '280.640'), (1, '278.210')] [2023-10-12 21:46:30,169][44958] Updated weights for policy 0, policy_version 42440 (0.0009) [2023-10-12 21:46:30,207][44959] Updated weights for policy 1, policy_version 42660 (0.0009) [2023-10-12 21:46:30,539][44958] Updated weights for policy 0, policy_version 42450 (0.0008) [2023-10-12 21:46:30,572][44959] Updated weights for policy 1, policy_version 42670 (0.0008) [2023-10-12 21:46:30,915][44958] Updated weights for policy 0, policy_version 42460 (0.0008) [2023-10-12 21:46:30,945][44959] Updated weights for policy 1, policy_version 42680 (0.0008) [2023-10-12 21:46:31,442][43579] Fps is (10 sec: 16384.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87195648. Throughput: 0: 1632.9, 1: 1638.6. Samples: 21803126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:31,443][43579] Avg episode reward: [(0, '282.090'), (1, '276.290')] [2023-10-12 21:46:35,038][44959] Updated weights for policy 1, policy_version 42690 (0.0008) [2023-10-12 21:46:35,110][44958] Updated weights for policy 0, policy_version 42470 (0.0008) [2023-10-12 21:46:35,404][44959] Updated weights for policy 1, policy_version 42700 (0.0007) [2023-10-12 21:46:35,490][44958] Updated weights for policy 0, policy_version 42480 (0.0008) [2023-10-12 21:46:35,775][44959] Updated weights for policy 1, policy_version 42710 (0.0009) [2023-10-12 21:46:35,859][44958] Updated weights for policy 0, policy_version 42490 (0.0009) [2023-10-12 21:46:36,139][44959] Updated weights for policy 1, policy_version 42720 (0.0008) [2023-10-12 21:46:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87261184. Throughput: 0: 1633.2, 1: 1642.2. Samples: 21814220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:36,443][43579] Avg episode reward: [(0, '283.930'), (1, '281.230')] [2023-10-12 21:46:40,207][44958] Updated weights for policy 0, policy_version 42500 (0.0008) [2023-10-12 21:46:40,460][44959] Updated weights for policy 1, policy_version 42730 (0.0007) [2023-10-12 21:46:40,584][44958] Updated weights for policy 0, policy_version 42510 (0.0007) [2023-10-12 21:46:40,827][44959] Updated weights for policy 1, policy_version 42740 (0.0009) [2023-10-12 21:46:40,969][44958] Updated weights for policy 0, policy_version 42520 (0.0007) [2023-10-12 21:46:41,198][44959] Updated weights for policy 1, policy_version 42750 (0.0008) [2023-10-12 21:46:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87326720. Throughput: 0: 1634.3, 1: 1638.4. Samples: 21833818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:41,444][43579] Avg episode reward: [(0, '274.600'), (1, '283.350')] [2023-10-12 21:46:45,231][44958] Updated weights for policy 0, policy_version 42530 (0.0007) [2023-10-12 21:46:45,347][44959] Updated weights for policy 1, policy_version 42760 (0.0009) [2023-10-12 21:46:45,597][44958] Updated weights for policy 0, policy_version 42540 (0.0007) [2023-10-12 21:46:45,708][44959] Updated weights for policy 1, policy_version 42770 (0.0009) [2023-10-12 21:46:45,973][44958] Updated weights for policy 0, policy_version 42550 (0.0007) [2023-10-12 21:46:46,079][44959] Updated weights for policy 1, policy_version 42780 (0.0007) [2023-10-12 21:46:46,341][44958] Updated weights for policy 0, policy_version 42560 (0.0008) [2023-10-12 21:46:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87392256. Throughput: 0: 1636.2, 1: 1640.6. Samples: 21852058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:46,443][43579] Avg episode reward: [(0, '277.540'), (1, '278.900')] [2023-10-12 21:46:50,324][44958] Updated weights for policy 0, policy_version 42570 (0.0008) [2023-10-12 21:46:50,363][44959] Updated weights for policy 1, policy_version 42790 (0.0008) [2023-10-12 21:46:50,696][44958] Updated weights for policy 0, policy_version 42580 (0.0008) [2023-10-12 21:46:50,733][44959] Updated weights for policy 1, policy_version 42800 (0.0009) [2023-10-12 21:46:51,063][44958] Updated weights for policy 0, policy_version 42590 (0.0008) [2023-10-12 21:46:51,099][44959] Updated weights for policy 1, policy_version 42810 (0.0008) [2023-10-12 21:46:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87457792. Throughput: 0: 1626.4, 1: 1634.9. Samples: 21862774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:51,443][43579] Avg episode reward: [(0, '275.290'), (1, '280.650')] [2023-10-12 21:46:55,099][44958] Updated weights for policy 0, policy_version 42600 (0.0010) [2023-10-12 21:46:55,214][44959] Updated weights for policy 1, policy_version 42820 (0.0007) [2023-10-12 21:46:55,480][44958] Updated weights for policy 0, policy_version 42610 (0.0007) [2023-10-12 21:46:55,576][44959] Updated weights for policy 1, policy_version 42830 (0.0009) [2023-10-12 21:46:55,853][44958] Updated weights for policy 0, policy_version 42620 (0.0008) [2023-10-12 21:46:55,945][44959] Updated weights for policy 1, policy_version 42840 (0.0008) [2023-10-12 21:46:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87523328. Throughput: 0: 1636.5, 1: 1636.6. Samples: 21883006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:46:56,443][43579] Avg episode reward: [(0, '273.960'), (1, '280.010')] [2023-10-12 21:47:00,005][44959] Updated weights for policy 1, policy_version 42850 (0.0008) [2023-10-12 21:47:00,069][44958] Updated weights for policy 0, policy_version 42630 (0.0008) [2023-10-12 21:47:00,381][44959] Updated weights for policy 1, policy_version 42860 (0.0009) [2023-10-12 21:47:00,430][44958] Updated weights for policy 0, policy_version 42640 (0.0008) [2023-10-12 21:47:00,747][44959] Updated weights for policy 1, policy_version 42870 (0.0007) [2023-10-12 21:47:00,801][44958] Updated weights for policy 0, policy_version 42650 (0.0008) [2023-10-12 21:47:01,120][44959] Updated weights for policy 1, policy_version 42880 (0.0009) [2023-10-12 21:47:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87588864. Throughput: 0: 1641.3, 1: 1634.8. Samples: 21901430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:47:01,444][43579] Avg episode reward: [(0, '271.930'), (1, '276.600')] [2023-10-12 21:47:05,003][44958] Updated weights for policy 0, policy_version 42660 (0.0009) [2023-10-12 21:47:05,336][44959] Updated weights for policy 1, policy_version 42890 (0.0007) [2023-10-12 21:47:05,368][44958] Updated weights for policy 0, policy_version 42670 (0.0009) [2023-10-12 21:47:05,700][44959] Updated weights for policy 1, policy_version 42900 (0.0008) [2023-10-12 21:47:05,737][44958] Updated weights for policy 0, policy_version 42680 (0.0010) [2023-10-12 21:47:06,069][44959] Updated weights for policy 1, policy_version 42910 (0.0010) [2023-10-12 21:47:06,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87654400. Throughput: 0: 1641.5, 1: 1641.2. Samples: 21912618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:47:06,444][43579] Avg episode reward: [(0, '263.180'), (1, '274.750')] [2023-10-12 21:47:09,711][44958] Updated weights for policy 0, policy_version 42690 (0.0010) [2023-10-12 21:47:10,091][44958] Updated weights for policy 0, policy_version 42700 (0.0009) [2023-10-12 21:47:10,155][44959] Updated weights for policy 1, policy_version 42920 (0.0008) [2023-10-12 21:47:10,462][44958] Updated weights for policy 0, policy_version 42710 (0.0007) [2023-10-12 21:47:10,517][44959] Updated weights for policy 1, policy_version 42930 (0.0007) [2023-10-12 21:47:10,828][44958] Updated weights for policy 0, policy_version 42720 (0.0011) [2023-10-12 21:47:10,887][44959] Updated weights for policy 1, policy_version 42940 (0.0007) [2023-10-12 21:47:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 87719936. Throughput: 0: 1638.7, 1: 1639.6. Samples: 21932224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:47:11,443][43579] Avg episode reward: [(0, '268.740'), (1, '274.410')] [2023-10-12 21:47:14,962][44959] Updated weights for policy 1, policy_version 42950 (0.0007) [2023-10-12 21:47:15,262][44958] Updated weights for policy 0, policy_version 42730 (0.0008) [2023-10-12 21:47:15,331][44959] Updated weights for policy 1, policy_version 42960 (0.0008) [2023-10-12 21:47:15,626][44958] Updated weights for policy 0, policy_version 42740 (0.0007) [2023-10-12 21:47:15,692][44959] Updated weights for policy 1, policy_version 42970 (0.0007) [2023-10-12 21:47:15,997][44958] Updated weights for policy 0, policy_version 42750 (0.0007) [2023-10-12 21:47:16,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 87785472. Throughput: 0: 1640.2, 1: 1643.6. Samples: 21950898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:47:16,443][43579] Avg episode reward: [(0, '270.930'), (1, '277.620')] [2023-10-12 21:47:20,075][44959] Updated weights for policy 1, policy_version 42980 (0.0009) [2023-10-12 21:47:20,153][44958] Updated weights for policy 0, policy_version 42760 (0.0007) [2023-10-12 21:47:20,444][44959] Updated weights for policy 1, policy_version 42990 (0.0008) [2023-10-12 21:47:20,518][44958] Updated weights for policy 0, policy_version 42770 (0.0009) [2023-10-12 21:47:20,807][44959] Updated weights for policy 1, policy_version 43000 (0.0009) [2023-10-12 21:47:20,895][44958] Updated weights for policy 0, policy_version 42780 (0.0010) [2023-10-12 21:47:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 87851008. Throughput: 0: 1639.5, 1: 1644.9. Samples: 21962022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:47:21,444][43579] Avg episode reward: [(0, '268.740'), (1, '273.660')] [2023-10-12 21:47:24,963][44958] Updated weights for policy 0, policy_version 42790 (0.0008) [2023-10-12 21:47:25,065][44959] Updated weights for policy 1, policy_version 43010 (0.0008) [2023-10-12 21:47:25,336][44958] Updated weights for policy 0, policy_version 42800 (0.0008) [2023-10-12 21:47:25,495][44959] Updated weights for policy 1, policy_version 43020 (0.0007) [2023-10-12 21:47:25,712][44958] Updated weights for policy 0, policy_version 42810 (0.0008) [2023-10-12 21:47:25,871][44959] Updated weights for policy 1, policy_version 43030 (0.0008) [2023-10-12 21:47:26,232][44959] Updated weights for policy 1, policy_version 43040 (0.0010) [2023-10-12 21:47:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 87916544. Throughput: 0: 1638.3, 1: 1647.0. Samples: 21981656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:47:26,444][43579] Avg episode reward: [(0, '267.090'), (1, '274.400')] [2023-10-12 21:47:29,910][44958] Updated weights for policy 0, policy_version 42820 (0.0008) [2023-10-12 21:47:30,287][44958] Updated weights for policy 0, policy_version 42830 (0.0009) [2023-10-12 21:47:30,458][44959] Updated weights for policy 1, policy_version 43050 (0.0009) [2023-10-12 21:47:30,643][44958] Updated weights for policy 0, policy_version 42840 (0.0008) [2023-10-12 21:47:30,829][44959] Updated weights for policy 1, policy_version 43060 (0.0008) [2023-10-12 21:47:31,193][44959] Updated weights for policy 1, policy_version 43070 (0.0008) [2023-10-12 21:47:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 87982080. Throughput: 0: 1637.9, 1: 1645.5. Samples: 21999808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:47:31,444][43579] Avg episode reward: [(0, '262.930'), (1, '272.530')] [2023-10-12 21:47:34,878][44958] Updated weights for policy 0, policy_version 42850 (0.0008) [2023-10-12 21:47:35,242][44958] Updated weights for policy 0, policy_version 42860 (0.0010) [2023-10-12 21:47:35,485][44959] Updated weights for policy 1, policy_version 43080 (0.0008) [2023-10-12 21:47:35,623][44958] Updated weights for policy 0, policy_version 42870 (0.0008) [2023-10-12 21:47:35,844][44959] Updated weights for policy 1, policy_version 43090 (0.0009) [2023-10-12 21:47:35,997][44958] Updated weights for policy 0, policy_version 42880 (0.0008) [2023-10-12 21:47:36,212][44959] Updated weights for policy 1, policy_version 43100 (0.0010) [2023-10-12 21:47:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88047616. Throughput: 0: 1644.6, 1: 1644.3. Samples: 22010772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:47:36,443][43579] Avg episode reward: [(0, '273.370'), (1, '271.800')] [2023-10-12 21:47:40,085][44959] Updated weights for policy 1, policy_version 43110 (0.0007) [2023-10-12 21:47:40,221][44958] Updated weights for policy 0, policy_version 42890 (0.0008) [2023-10-12 21:47:40,448][44959] Updated weights for policy 1, policy_version 43120 (0.0009) [2023-10-12 21:47:40,594][44958] Updated weights for policy 0, policy_version 42900 (0.0009) [2023-10-12 21:47:40,812][44959] Updated weights for policy 1, policy_version 43130 (0.0008) [2023-10-12 21:47:40,968][44958] Updated weights for policy 0, policy_version 42910 (0.0008) [2023-10-12 21:47:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88113152. Throughput: 0: 1636.8, 1: 1644.3. Samples: 22030660. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:47:41,444][43579] Avg episode reward: [(0, '269.990'), (1, '277.720')] [2023-10-12 21:47:44,817][44959] Updated weights for policy 1, policy_version 43140 (0.0008) [2023-10-12 21:47:45,177][44959] Updated weights for policy 1, policy_version 43150 (0.0007) [2023-10-12 21:47:45,265][44958] Updated weights for policy 0, policy_version 42920 (0.0009) [2023-10-12 21:47:45,539][44959] Updated weights for policy 1, policy_version 43160 (0.0008) [2023-10-12 21:47:45,631][44958] Updated weights for policy 0, policy_version 42930 (0.0009) [2023-10-12 21:47:46,009][44958] Updated weights for policy 0, policy_version 42940 (0.0008) [2023-10-12 21:47:46,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88178688. Throughput: 0: 1635.5, 1: 1645.4. Samples: 22049070. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:47:46,444][43579] Avg episode reward: [(0, '266.640'), (1, '275.490')] [2023-10-12 21:47:46,456][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000043168_44204032.pth... [2023-10-12 21:47:46,457][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000042944_43974656.pth... [2023-10-12 21:47:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000041408_42401792.pth [2023-10-12 21:47:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000041632_42631168.pth [2023-10-12 21:47:49,794][44959] Updated weights for policy 1, policy_version 43170 (0.0008) [2023-10-12 21:47:50,123][44958] Updated weights for policy 0, policy_version 42950 (0.0009) [2023-10-12 21:47:50,160][44959] Updated weights for policy 1, policy_version 43180 (0.0007) [2023-10-12 21:47:50,489][44958] Updated weights for policy 0, policy_version 42960 (0.0009) [2023-10-12 21:47:50,522][44959] Updated weights for policy 1, policy_version 43190 (0.0008) [2023-10-12 21:47:50,862][44958] Updated weights for policy 0, policy_version 42970 (0.0009) [2023-10-12 21:47:50,889][44959] Updated weights for policy 1, policy_version 43200 (0.0008) [2023-10-12 21:47:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88244224. Throughput: 0: 1632.4, 1: 1645.0. Samples: 22060098. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:47:51,444][43579] Avg episode reward: [(0, '267.940'), (1, '273.010')] [2023-10-12 21:47:54,957][44959] Updated weights for policy 1, policy_version 43210 (0.0007) [2023-10-12 21:47:55,045][44958] Updated weights for policy 0, policy_version 42980 (0.0010) [2023-10-12 21:47:55,325][44959] Updated weights for policy 1, policy_version 43220 (0.0007) [2023-10-12 21:47:55,420][44958] Updated weights for policy 0, policy_version 42990 (0.0008) [2023-10-12 21:47:55,683][44959] Updated weights for policy 1, policy_version 43230 (0.0007) [2023-10-12 21:47:55,790][44958] Updated weights for policy 0, policy_version 43000 (0.0007) [2023-10-12 21:47:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88309760. Throughput: 0: 1635.5, 1: 1644.1. Samples: 22079806. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:47:56,443][43579] Avg episode reward: [(0, '270.450'), (1, '270.140')] [2023-10-12 21:47:59,940][44959] Updated weights for policy 1, policy_version 43240 (0.0009) [2023-10-12 21:48:00,060][44958] Updated weights for policy 0, policy_version 43010 (0.0010) [2023-10-12 21:48:00,306][44959] Updated weights for policy 1, policy_version 43250 (0.0010) [2023-10-12 21:48:00,423][44958] Updated weights for policy 0, policy_version 43020 (0.0008) [2023-10-12 21:48:00,671][44959] Updated weights for policy 1, policy_version 43260 (0.0009) [2023-10-12 21:48:00,800][44958] Updated weights for policy 0, policy_version 43030 (0.0008) [2023-10-12 21:48:01,170][44958] Updated weights for policy 0, policy_version 43040 (0.0008) [2023-10-12 21:48:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88375296. Throughput: 0: 1634.3, 1: 1642.1. Samples: 22098336. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:48:01,444][43579] Avg episode reward: [(0, '271.630'), (1, '273.780')] [2023-10-12 21:48:04,874][44959] Updated weights for policy 1, policy_version 43270 (0.0009) [2023-10-12 21:48:05,235][44959] Updated weights for policy 1, policy_version 43280 (0.0007) [2023-10-12 21:48:05,466][44958] Updated weights for policy 0, policy_version 43050 (0.0010) [2023-10-12 21:48:05,606][44959] Updated weights for policy 1, policy_version 43290 (0.0008) [2023-10-12 21:48:05,841][44958] Updated weights for policy 0, policy_version 43060 (0.0009) [2023-10-12 21:48:06,218][44958] Updated weights for policy 0, policy_version 43070 (0.0009) [2023-10-12 21:48:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88440832. Throughput: 0: 1629.1, 1: 1644.7. Samples: 22109342. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-12 21:48:06,444][43579] Avg episode reward: [(0, '272.370'), (1, '272.180')] [2023-10-12 21:48:09,835][44959] Updated weights for policy 1, policy_version 43300 (0.0009) [2023-10-12 21:48:10,233][44959] Updated weights for policy 1, policy_version 43310 (0.0009) [2023-10-12 21:48:10,350][44958] Updated weights for policy 0, policy_version 43080 (0.0009) [2023-10-12 21:48:10,599][44959] Updated weights for policy 1, policy_version 43320 (0.0007) [2023-10-12 21:48:10,720][44958] Updated weights for policy 0, policy_version 43090 (0.0010) [2023-10-12 21:48:11,090][44958] Updated weights for policy 0, policy_version 43100 (0.0009) [2023-10-12 21:48:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88506368. Throughput: 0: 1632.1, 1: 1636.9. Samples: 22128764. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-12 21:48:11,443][43579] Avg episode reward: [(0, '279.680'), (1, '273.980')] [2023-10-12 21:48:14,818][44959] Updated weights for policy 1, policy_version 43330 (0.0008) [2023-10-12 21:48:15,197][44959] Updated weights for policy 1, policy_version 43340 (0.0007) [2023-10-12 21:48:15,391][44958] Updated weights for policy 0, policy_version 43110 (0.0009) [2023-10-12 21:48:15,565][44959] Updated weights for policy 1, policy_version 43350 (0.0008) [2023-10-12 21:48:15,767][44958] Updated weights for policy 0, policy_version 43120 (0.0008) [2023-10-12 21:48:15,935][44959] Updated weights for policy 1, policy_version 43360 (0.0007) [2023-10-12 21:48:16,129][44958] Updated weights for policy 0, policy_version 43130 (0.0009) [2023-10-12 21:48:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88571904. Throughput: 0: 1632.1, 1: 1639.0. Samples: 22147006. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-12 21:48:16,443][43579] Avg episode reward: [(0, '281.460'), (1, '273.370')] [2023-10-12 21:48:19,927][44959] Updated weights for policy 1, policy_version 43370 (0.0007) [2023-10-12 21:48:20,187][44958] Updated weights for policy 0, policy_version 43140 (0.0009) [2023-10-12 21:48:20,295][44959] Updated weights for policy 1, policy_version 43380 (0.0007) [2023-10-12 21:48:20,549][44958] Updated weights for policy 0, policy_version 43150 (0.0008) [2023-10-12 21:48:20,659][44959] Updated weights for policy 1, policy_version 43390 (0.0007) [2023-10-12 21:48:20,925][44958] Updated weights for policy 0, policy_version 43160 (0.0009) [2023-10-12 21:48:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88637440. Throughput: 0: 1628.5, 1: 1649.3. Samples: 22158276. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-12 21:48:21,443][43579] Avg episode reward: [(0, '284.890'), (1, '282.880')] [2023-10-12 21:48:21,444][44518] Saving new best policy, reward=284.890! [2023-10-12 21:48:24,722][44959] Updated weights for policy 1, policy_version 43400 (0.0010) [2023-10-12 21:48:25,096][44959] Updated weights for policy 1, policy_version 43410 (0.0010) [2023-10-12 21:48:25,190][44958] Updated weights for policy 0, policy_version 43170 (0.0008) [2023-10-12 21:48:25,454][44959] Updated weights for policy 1, policy_version 43420 (0.0009) [2023-10-12 21:48:25,563][44958] Updated weights for policy 0, policy_version 43180 (0.0009) [2023-10-12 21:48:25,948][44958] Updated weights for policy 0, policy_version 43190 (0.0008) [2023-10-12 21:48:26,311][44958] Updated weights for policy 0, policy_version 43200 (0.0010) [2023-10-12 21:48:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 88702976. Throughput: 0: 1632.7, 1: 1640.5. Samples: 22177954. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-12 21:48:26,443][43579] Avg episode reward: [(0, '285.910'), (1, '284.290')] [2023-10-12 21:48:26,444][44518] Saving new best policy, reward=285.910! [2023-10-12 21:48:29,851][44959] Updated weights for policy 1, policy_version 43430 (0.0009) [2023-10-12 21:48:30,235][44959] Updated weights for policy 1, policy_version 43440 (0.0009) [2023-10-12 21:48:30,476][44958] Updated weights for policy 0, policy_version 43210 (0.0007) [2023-10-12 21:48:30,597][44959] Updated weights for policy 1, policy_version 43450 (0.0008) [2023-10-12 21:48:30,847][44958] Updated weights for policy 0, policy_version 43220 (0.0007) [2023-10-12 21:48:31,216][44958] Updated weights for policy 0, policy_version 43230 (0.0007) [2023-10-12 21:48:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 88768512. Throughput: 0: 1630.0, 1: 1647.4. Samples: 22196554. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-12 21:48:31,443][43579] Avg episode reward: [(0, '281.150'), (1, '282.800')] [2023-10-12 21:48:34,811][44959] Updated weights for policy 1, policy_version 43460 (0.0008) [2023-10-12 21:48:35,187][44959] Updated weights for policy 1, policy_version 43470 (0.0007) [2023-10-12 21:48:35,304][44958] Updated weights for policy 0, policy_version 43240 (0.0008) [2023-10-12 21:48:35,546][44959] Updated weights for policy 1, policy_version 43480 (0.0007) [2023-10-12 21:48:35,674][44958] Updated weights for policy 0, policy_version 43250 (0.0010) [2023-10-12 21:48:36,051][44958] Updated weights for policy 0, policy_version 43260 (0.0010) [2023-10-12 21:48:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 88834048. Throughput: 0: 1631.0, 1: 1648.4. Samples: 22207668. Policy #0 lag: (min: 31.0, avg: 32.8, max: 60.0) [2023-10-12 21:48:36,443][43579] Avg episode reward: [(0, '282.040'), (1, '277.970')] [2023-10-12 21:48:39,591][44959] Updated weights for policy 1, policy_version 43490 (0.0009) [2023-10-12 21:48:39,954][44959] Updated weights for policy 1, policy_version 43500 (0.0008) [2023-10-12 21:48:40,322][44959] Updated weights for policy 1, policy_version 43510 (0.0007) [2023-10-12 21:48:40,437][44958] Updated weights for policy 0, policy_version 43270 (0.0009) [2023-10-12 21:48:40,688][44959] Updated weights for policy 1, policy_version 43520 (0.0007) [2023-10-12 21:48:40,804][44958] Updated weights for policy 0, policy_version 43280 (0.0010) [2023-10-12 21:48:41,183][44958] Updated weights for policy 0, policy_version 43290 (0.0008) [2023-10-12 21:48:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 88899584. Throughput: 0: 1635.2, 1: 1641.2. Samples: 22227244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:48:41,443][43579] Avg episode reward: [(0, '282.440'), (1, '277.020')] [2023-10-12 21:48:44,921][44959] Updated weights for policy 1, policy_version 43530 (0.0008) [2023-10-12 21:48:45,282][44959] Updated weights for policy 1, policy_version 43540 (0.0009) [2023-10-12 21:48:45,370][44958] Updated weights for policy 0, policy_version 43300 (0.0009) [2023-10-12 21:48:45,653][44959] Updated weights for policy 1, policy_version 43550 (0.0007) [2023-10-12 21:48:45,736][44958] Updated weights for policy 0, policy_version 43310 (0.0007) [2023-10-12 21:48:46,112][44958] Updated weights for policy 0, policy_version 43320 (0.0008) [2023-10-12 21:48:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 88965120. Throughput: 0: 1633.3, 1: 1647.7. Samples: 22245980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:48:46,443][43579] Avg episode reward: [(0, '281.630'), (1, '275.420')] [2023-10-12 21:48:49,900][44959] Updated weights for policy 1, policy_version 43560 (0.0007) [2023-10-12 21:48:50,265][44959] Updated weights for policy 1, policy_version 43570 (0.0007) [2023-10-12 21:48:50,273][44958] Updated weights for policy 0, policy_version 43330 (0.0007) [2023-10-12 21:48:50,627][44959] Updated weights for policy 1, policy_version 43580 (0.0008) [2023-10-12 21:48:50,661][44958] Updated weights for policy 0, policy_version 43340 (0.0007) [2023-10-12 21:48:51,029][44958] Updated weights for policy 0, policy_version 43350 (0.0009) [2023-10-12 21:48:51,402][44958] Updated weights for policy 0, policy_version 43360 (0.0008) [2023-10-12 21:48:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 89030656. Throughput: 0: 1631.4, 1: 1645.0. Samples: 22256778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:48:51,443][43579] Avg episode reward: [(0, '280.300'), (1, '275.570')] [2023-10-12 21:48:54,868][44959] Updated weights for policy 1, policy_version 43590 (0.0007) [2023-10-12 21:48:55,229][44959] Updated weights for policy 1, policy_version 43600 (0.0009) [2023-10-12 21:48:55,591][44959] Updated weights for policy 1, policy_version 43610 (0.0008) [2023-10-12 21:48:55,591][44958] Updated weights for policy 0, policy_version 43370 (0.0007) [2023-10-12 21:48:55,967][44958] Updated weights for policy 0, policy_version 43380 (0.0008) [2023-10-12 21:48:56,335][44958] Updated weights for policy 0, policy_version 43390 (0.0007) [2023-10-12 21:48:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 89096192. Throughput: 0: 1637.3, 1: 1646.8. Samples: 22276546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:48:56,444][43579] Avg episode reward: [(0, '282.190'), (1, '272.060')] [2023-10-12 21:48:59,839][44959] Updated weights for policy 1, policy_version 43620 (0.0007) [2023-10-12 21:49:00,207][44959] Updated weights for policy 1, policy_version 43630 (0.0010) [2023-10-12 21:49:00,417][44958] Updated weights for policy 0, policy_version 43400 (0.0009) [2023-10-12 21:49:00,571][44959] Updated weights for policy 1, policy_version 43640 (0.0008) [2023-10-12 21:49:00,805][44958] Updated weights for policy 0, policy_version 43410 (0.0009) [2023-10-12 21:49:01,172][44958] Updated weights for policy 0, policy_version 43420 (0.0010) [2023-10-12 21:49:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 89161728. Throughput: 0: 1637.3, 1: 1649.3. Samples: 22294904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:49:01,443][43579] Avg episode reward: [(0, '279.460'), (1, '273.180')] [2023-10-12 21:49:04,588][44959] Updated weights for policy 1, policy_version 43650 (0.0007) [2023-10-12 21:49:04,957][44959] Updated weights for policy 1, policy_version 43660 (0.0009) [2023-10-12 21:49:05,328][44959] Updated weights for policy 1, policy_version 43670 (0.0008) [2023-10-12 21:49:05,370][44958] Updated weights for policy 0, policy_version 43430 (0.0009) [2023-10-12 21:49:05,701][44959] Updated weights for policy 1, policy_version 43680 (0.0010) [2023-10-12 21:49:05,755][44958] Updated weights for policy 0, policy_version 43440 (0.0009) [2023-10-12 21:49:06,126][44958] Updated weights for policy 0, policy_version 43450 (0.0008) [2023-10-12 21:49:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 89227264. Throughput: 0: 1636.5, 1: 1648.2. Samples: 22306090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:49:06,443][43579] Avg episode reward: [(0, '278.640'), (1, '274.040')] [2023-10-12 21:49:10,026][44959] Updated weights for policy 1, policy_version 43690 (0.0009) [2023-10-12 21:49:10,165][44958] Updated weights for policy 0, policy_version 43460 (0.0007) [2023-10-12 21:49:10,395][44959] Updated weights for policy 1, policy_version 43700 (0.0007) [2023-10-12 21:49:10,546][44958] Updated weights for policy 0, policy_version 43470 (0.0008) [2023-10-12 21:49:10,766][44959] Updated weights for policy 1, policy_version 43710 (0.0008) [2023-10-12 21:49:10,913][44958] Updated weights for policy 0, policy_version 43480 (0.0007) [2023-10-12 21:49:11,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 89292800. Throughput: 0: 1635.6, 1: 1645.3. Samples: 22325592. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-12 21:49:11,444][43579] Avg episode reward: [(0, '269.260'), (1, '273.320')] [2023-10-12 21:49:14,977][44959] Updated weights for policy 1, policy_version 43720 (0.0008) [2023-10-12 21:49:15,087][44958] Updated weights for policy 0, policy_version 43490 (0.0007) [2023-10-12 21:49:15,364][44959] Updated weights for policy 1, policy_version 43730 (0.0011) [2023-10-12 21:49:15,451][44958] Updated weights for policy 0, policy_version 43500 (0.0008) [2023-10-12 21:49:15,723][44959] Updated weights for policy 1, policy_version 43740 (0.0008) [2023-10-12 21:49:15,831][44958] Updated weights for policy 0, policy_version 43510 (0.0008) [2023-10-12 21:49:16,203][44958] Updated weights for policy 0, policy_version 43520 (0.0009) [2023-10-12 21:49:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 89358336. Throughput: 0: 1640.4, 1: 1638.4. Samples: 22344100. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-12 21:49:16,444][43579] Avg episode reward: [(0, '270.300'), (1, '277.000')] [2023-10-12 21:49:19,906][44959] Updated weights for policy 1, policy_version 43750 (0.0010) [2023-10-12 21:49:20,273][44959] Updated weights for policy 1, policy_version 43760 (0.0008) [2023-10-12 21:49:20,527][44958] Updated weights for policy 0, policy_version 43530 (0.0008) [2023-10-12 21:49:20,641][44959] Updated weights for policy 1, policy_version 43770 (0.0010) [2023-10-12 21:49:20,893][44958] Updated weights for policy 0, policy_version 43540 (0.0007) [2023-10-12 21:49:21,273][44958] Updated weights for policy 0, policy_version 43550 (0.0007) [2023-10-12 21:49:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 89423872. Throughput: 0: 1638.9, 1: 1645.0. Samples: 22355446. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-12 21:49:21,444][43579] Avg episode reward: [(0, '267.570'), (1, '275.120')] [2023-10-12 21:49:24,730][44959] Updated weights for policy 1, policy_version 43780 (0.0009) [2023-10-12 21:49:25,099][44959] Updated weights for policy 1, policy_version 43790 (0.0008) [2023-10-12 21:49:25,147][44958] Updated weights for policy 0, policy_version 43560 (0.0008) [2023-10-12 21:49:25,462][44959] Updated weights for policy 1, policy_version 43800 (0.0008) [2023-10-12 21:49:25,529][44958] Updated weights for policy 0, policy_version 43570 (0.0008) [2023-10-12 21:49:25,896][44958] Updated weights for policy 0, policy_version 43580 (0.0008) [2023-10-12 21:49:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 89489408. Throughput: 0: 1638.5, 1: 1640.7. Samples: 22374810. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-12 21:49:26,443][43579] Avg episode reward: [(0, '267.590'), (1, '270.070')] [2023-10-12 21:49:29,668][44959] Updated weights for policy 1, policy_version 43810 (0.0007) [2023-10-12 21:49:30,031][44959] Updated weights for policy 1, policy_version 43820 (0.0008) [2023-10-12 21:49:30,213][44958] Updated weights for policy 0, policy_version 43590 (0.0007) [2023-10-12 21:49:30,398][44959] Updated weights for policy 1, policy_version 43830 (0.0008) [2023-10-12 21:49:30,582][44958] Updated weights for policy 0, policy_version 43600 (0.0007) [2023-10-12 21:49:30,765][44959] Updated weights for policy 1, policy_version 43840 (0.0009) [2023-10-12 21:49:30,963][44958] Updated weights for policy 0, policy_version 43610 (0.0009) [2023-10-12 21:49:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 89554944. Throughput: 0: 1639.8, 1: 1635.2. Samples: 22393356. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-12 21:49:31,444][43579] Avg episode reward: [(0, '267.840'), (1, '266.130')] [2023-10-12 21:49:34,985][44958] Updated weights for policy 0, policy_version 43620 (0.0007) [2023-10-12 21:49:35,055][44959] Updated weights for policy 1, policy_version 43850 (0.0010) [2023-10-12 21:49:35,363][44958] Updated weights for policy 0, policy_version 43630 (0.0007) [2023-10-12 21:49:35,425][44959] Updated weights for policy 1, policy_version 43860 (0.0008) [2023-10-12 21:49:35,743][44958] Updated weights for policy 0, policy_version 43640 (0.0009) [2023-10-12 21:49:35,792][44959] Updated weights for policy 1, policy_version 43870 (0.0008) [2023-10-12 21:49:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 89620480. Throughput: 0: 1645.6, 1: 1639.6. Samples: 22404616. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-12 21:49:36,443][43579] Avg episode reward: [(0, '266.640'), (1, '267.140')] [2023-10-12 21:49:39,828][44958] Updated weights for policy 0, policy_version 43650 (0.0008) [2023-10-12 21:49:39,974][44959] Updated weights for policy 1, policy_version 43880 (0.0007) [2023-10-12 21:49:40,214][44958] Updated weights for policy 0, policy_version 43660 (0.0010) [2023-10-12 21:49:40,353][44959] Updated weights for policy 1, policy_version 43890 (0.0007) [2023-10-12 21:49:40,587][44958] Updated weights for policy 0, policy_version 43670 (0.0009) [2023-10-12 21:49:40,724][44959] Updated weights for policy 1, policy_version 43900 (0.0008) [2023-10-12 21:49:40,954][44958] Updated weights for policy 0, policy_version 43680 (0.0011) [2023-10-12 21:49:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 89686016. Throughput: 0: 1638.6, 1: 1633.7. Samples: 22423800. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-12 21:49:41,444][43579] Avg episode reward: [(0, '272.600'), (1, '271.370')] [2023-10-12 21:49:44,975][44959] Updated weights for policy 1, policy_version 43910 (0.0009) [2023-10-12 21:49:45,164][44958] Updated weights for policy 0, policy_version 43690 (0.0008) [2023-10-12 21:49:45,349][44959] Updated weights for policy 1, policy_version 43920 (0.0007) [2023-10-12 21:49:45,535][44958] Updated weights for policy 0, policy_version 43700 (0.0009) [2023-10-12 21:49:45,706][44959] Updated weights for policy 1, policy_version 43930 (0.0007) [2023-10-12 21:49:45,909][44958] Updated weights for policy 0, policy_version 43710 (0.0008) [2023-10-12 21:49:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 89751552. Throughput: 0: 1641.2, 1: 1632.9. Samples: 22442240. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-12 21:49:46,444][43579] Avg episode reward: [(0, '271.350'), (1, '272.920')] [2023-10-12 21:49:46,456][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000043712_44761088.pth... [2023-10-12 21:49:46,457][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000043936_44990464.pth... [2023-10-12 21:49:46,495][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000042176_43188224.pth [2023-10-12 21:49:46,501][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000042400_43417600.pth [2023-10-12 21:49:49,911][44959] Updated weights for policy 1, policy_version 43940 (0.0008) [2023-10-12 21:49:50,159][44958] Updated weights for policy 0, policy_version 43720 (0.0008) [2023-10-12 21:49:50,280][44959] Updated weights for policy 1, policy_version 43950 (0.0009) [2023-10-12 21:49:50,532][44958] Updated weights for policy 0, policy_version 43730 (0.0008) [2023-10-12 21:49:50,646][44959] Updated weights for policy 1, policy_version 43960 (0.0007) [2023-10-12 21:49:50,893][44958] Updated weights for policy 0, policy_version 43740 (0.0008) [2023-10-12 21:49:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 89817088. Throughput: 0: 1643.5, 1: 1631.4. Samples: 22453458. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-12 21:49:51,444][43579] Avg episode reward: [(0, '271.960'), (1, '273.270')] [2023-10-12 21:49:54,843][44959] Updated weights for policy 1, policy_version 43970 (0.0009) [2023-10-12 21:49:55,108][44958] Updated weights for policy 0, policy_version 43750 (0.0007) [2023-10-12 21:49:55,210][44959] Updated weights for policy 1, policy_version 43980 (0.0007) [2023-10-12 21:49:55,482][44958] Updated weights for policy 0, policy_version 43760 (0.0007) [2023-10-12 21:49:55,582][44959] Updated weights for policy 1, policy_version 43990 (0.0008) [2023-10-12 21:49:55,844][44958] Updated weights for policy 0, policy_version 43770 (0.0007) [2023-10-12 21:49:55,942][44959] Updated weights for policy 1, policy_version 44000 (0.0007) [2023-10-12 21:49:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 89882624. Throughput: 0: 1640.5, 1: 1634.6. Samples: 22472972. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-12 21:49:56,444][43579] Avg episode reward: [(0, '273.320'), (1, '279.290')] [2023-10-12 21:49:59,941][44958] Updated weights for policy 0, policy_version 43780 (0.0008) [2023-10-12 21:50:00,130][44959] Updated weights for policy 1, policy_version 44010 (0.0008) [2023-10-12 21:50:00,312][44958] Updated weights for policy 0, policy_version 43790 (0.0009) [2023-10-12 21:50:00,496][44959] Updated weights for policy 1, policy_version 44020 (0.0009) [2023-10-12 21:50:00,686][44958] Updated weights for policy 0, policy_version 43800 (0.0008) [2023-10-12 21:50:00,870][44959] Updated weights for policy 1, policy_version 44030 (0.0008) [2023-10-12 21:50:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 89948160. Throughput: 0: 1636.4, 1: 1633.4. Samples: 22491242. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-12 21:50:01,444][43579] Avg episode reward: [(0, '273.310'), (1, '284.250')] [2023-10-12 21:50:04,815][44958] Updated weights for policy 0, policy_version 43810 (0.0010) [2023-10-12 21:50:05,064][44959] Updated weights for policy 1, policy_version 44040 (0.0008) [2023-10-12 21:50:05,195][44958] Updated weights for policy 0, policy_version 43820 (0.0008) [2023-10-12 21:50:05,427][44959] Updated weights for policy 1, policy_version 44050 (0.0008) [2023-10-12 21:50:05,569][44958] Updated weights for policy 0, policy_version 43830 (0.0007) [2023-10-12 21:50:05,794][44959] Updated weights for policy 1, policy_version 44060 (0.0008) [2023-10-12 21:50:05,937][44958] Updated weights for policy 0, policy_version 43840 (0.0008) [2023-10-12 21:50:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90013696. Throughput: 0: 1640.6, 1: 1627.1. Samples: 22502490. Policy #0 lag: (min: 25.0, avg: 28.2, max: 57.0) [2023-10-12 21:50:06,443][43579] Avg episode reward: [(0, '276.020'), (1, '284.320')] [2023-10-12 21:50:09,905][44959] Updated weights for policy 1, policy_version 44070 (0.0009) [2023-10-12 21:50:10,117][44958] Updated weights for policy 0, policy_version 43850 (0.0010) [2023-10-12 21:50:10,277][44959] Updated weights for policy 1, policy_version 44080 (0.0007) [2023-10-12 21:50:10,494][44958] Updated weights for policy 0, policy_version 43860 (0.0008) [2023-10-12 21:50:10,643][44959] Updated weights for policy 1, policy_version 44090 (0.0007) [2023-10-12 21:50:10,868][44958] Updated weights for policy 0, policy_version 43870 (0.0008) [2023-10-12 21:50:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90079232. Throughput: 0: 1639.6, 1: 1634.6. Samples: 22522148. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:50:11,444][43579] Avg episode reward: [(0, '268.880'), (1, '282.180')] [2023-10-12 21:50:14,699][44959] Updated weights for policy 1, policy_version 44100 (0.0008) [2023-10-12 21:50:15,075][44959] Updated weights for policy 1, policy_version 44110 (0.0009) [2023-10-12 21:50:15,439][44959] Updated weights for policy 1, policy_version 44120 (0.0007) [2023-10-12 21:50:15,444][44958] Updated weights for policy 0, policy_version 43880 (0.0009) [2023-10-12 21:50:15,825][44958] Updated weights for policy 0, policy_version 43890 (0.0008) [2023-10-12 21:50:16,180][44958] Updated weights for policy 0, policy_version 43900 (0.0008) [2023-10-12 21:50:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90144768. Throughput: 0: 1643.9, 1: 1638.5. Samples: 22541062. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:50:16,444][43579] Avg episode reward: [(0, '274.420'), (1, '282.050')] [2023-10-12 21:50:19,862][44959] Updated weights for policy 1, policy_version 44130 (0.0009) [2023-10-12 21:50:20,226][44959] Updated weights for policy 1, policy_version 44140 (0.0009) [2023-10-12 21:50:20,351][44958] Updated weights for policy 0, policy_version 43910 (0.0009) [2023-10-12 21:50:20,604][44959] Updated weights for policy 1, policy_version 44150 (0.0009) [2023-10-12 21:50:20,722][44958] Updated weights for policy 0, policy_version 43920 (0.0009) [2023-10-12 21:50:20,973][44959] Updated weights for policy 1, policy_version 44160 (0.0007) [2023-10-12 21:50:21,093][44958] Updated weights for policy 0, policy_version 43930 (0.0009) [2023-10-12 21:50:21,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90210304. Throughput: 0: 1639.6, 1: 1634.2. Samples: 22551938. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:50:21,443][43579] Avg episode reward: [(0, '271.140'), (1, '284.490')] [2023-10-12 21:50:25,157][44959] Updated weights for policy 1, policy_version 44170 (0.0009) [2023-10-12 21:50:25,281][44958] Updated weights for policy 0, policy_version 43940 (0.0011) [2023-10-12 21:50:25,525][44959] Updated weights for policy 1, policy_version 44180 (0.0008) [2023-10-12 21:50:25,655][44958] Updated weights for policy 0, policy_version 43950 (0.0009) [2023-10-12 21:50:25,899][44959] Updated weights for policy 1, policy_version 44190 (0.0008) [2023-10-12 21:50:26,037][44958] Updated weights for policy 0, policy_version 43960 (0.0010) [2023-10-12 21:50:26,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90275840. Throughput: 0: 1645.1, 1: 1637.1. Samples: 22571498. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:50:26,443][43579] Avg episode reward: [(0, '266.160'), (1, '283.850')] [2023-10-12 21:50:29,896][44959] Updated weights for policy 1, policy_version 44200 (0.0008) [2023-10-12 21:50:30,170][44958] Updated weights for policy 0, policy_version 43970 (0.0007) [2023-10-12 21:50:30,262][44959] Updated weights for policy 1, policy_version 44210 (0.0009) [2023-10-12 21:50:30,539][44958] Updated weights for policy 0, policy_version 43980 (0.0007) [2023-10-12 21:50:30,634][44959] Updated weights for policy 1, policy_version 44220 (0.0008) [2023-10-12 21:50:30,909][44958] Updated weights for policy 0, policy_version 43990 (0.0010) [2023-10-12 21:50:31,285][44958] Updated weights for policy 0, policy_version 44000 (0.0010) [2023-10-12 21:50:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90341376. Throughput: 0: 1638.7, 1: 1644.0. Samples: 22589964. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:50:31,444][43579] Avg episode reward: [(0, '263.300'), (1, '280.150')] [2023-10-12 21:50:34,813][44959] Updated weights for policy 1, policy_version 44230 (0.0007) [2023-10-12 21:50:35,180][44959] Updated weights for policy 1, policy_version 44240 (0.0007) [2023-10-12 21:50:35,373][44958] Updated weights for policy 0, policy_version 44010 (0.0007) [2023-10-12 21:50:35,546][44959] Updated weights for policy 1, policy_version 44250 (0.0008) [2023-10-12 21:50:35,737][44958] Updated weights for policy 0, policy_version 44020 (0.0008) [2023-10-12 21:50:36,115][44958] Updated weights for policy 0, policy_version 44030 (0.0009) [2023-10-12 21:50:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90406912. Throughput: 0: 1641.1, 1: 1645.3. Samples: 22601346. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 21:50:36,443][43579] Avg episode reward: [(0, '267.160'), (1, '275.870')] [2023-10-12 21:50:39,753][44959] Updated weights for policy 1, policy_version 44260 (0.0008) [2023-10-12 21:50:40,128][44959] Updated weights for policy 1, policy_version 44270 (0.0008) [2023-10-12 21:50:40,250][44958] Updated weights for policy 0, policy_version 44040 (0.0009) [2023-10-12 21:50:40,493][44959] Updated weights for policy 1, policy_version 44280 (0.0009) [2023-10-12 21:50:40,615][44958] Updated weights for policy 0, policy_version 44050 (0.0008) [2023-10-12 21:50:40,996][44958] Updated weights for policy 0, policy_version 44060 (0.0007) [2023-10-12 21:50:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90472448. Throughput: 0: 1644.0, 1: 1641.5. Samples: 22620816. Policy #0 lag: (min: 4.0, avg: 7.6, max: 36.0) [2023-10-12 21:50:41,444][43579] Avg episode reward: [(0, '269.840'), (1, '275.380')] [2023-10-12 21:50:44,529][44959] Updated weights for policy 1, policy_version 44290 (0.0008) [2023-10-12 21:50:44,891][44959] Updated weights for policy 1, policy_version 44300 (0.0008) [2023-10-12 21:50:45,262][44959] Updated weights for policy 1, policy_version 44310 (0.0009) [2023-10-12 21:50:45,272][44958] Updated weights for policy 0, policy_version 44070 (0.0009) [2023-10-12 21:50:45,628][44959] Updated weights for policy 1, policy_version 44320 (0.0007) [2023-10-12 21:50:45,643][44958] Updated weights for policy 0, policy_version 44080 (0.0009) [2023-10-12 21:50:46,017][44958] Updated weights for policy 0, policy_version 44090 (0.0008) [2023-10-12 21:50:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90537984. Throughput: 0: 1644.5, 1: 1647.7. Samples: 22639394. Policy #0 lag: (min: 4.0, avg: 7.6, max: 36.0) [2023-10-12 21:50:46,443][43579] Avg episode reward: [(0, '274.540'), (1, '272.930')] [2023-10-12 21:50:49,729][44959] Updated weights for policy 1, policy_version 44330 (0.0009) [2023-10-12 21:50:50,095][44959] Updated weights for policy 1, policy_version 44340 (0.0009) [2023-10-12 21:50:50,190][44958] Updated weights for policy 0, policy_version 44100 (0.0010) [2023-10-12 21:50:50,469][44959] Updated weights for policy 1, policy_version 44350 (0.0008) [2023-10-12 21:50:50,560][44958] Updated weights for policy 0, policy_version 44110 (0.0008) [2023-10-12 21:50:50,931][44958] Updated weights for policy 0, policy_version 44120 (0.0009) [2023-10-12 21:50:51,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90603520. Throughput: 0: 1639.9, 1: 1651.3. Samples: 22650596. Policy #0 lag: (min: 4.0, avg: 7.6, max: 36.0) [2023-10-12 21:50:51,443][43579] Avg episode reward: [(0, '274.950'), (1, '267.780')] [2023-10-12 21:50:54,629][44959] Updated weights for policy 1, policy_version 44360 (0.0008) [2023-10-12 21:50:54,997][44959] Updated weights for policy 1, policy_version 44370 (0.0008) [2023-10-12 21:50:55,010][44958] Updated weights for policy 0, policy_version 44130 (0.0009) [2023-10-12 21:50:55,368][44959] Updated weights for policy 1, policy_version 44380 (0.0008) [2023-10-12 21:50:55,390][44958] Updated weights for policy 0, policy_version 44140 (0.0008) [2023-10-12 21:50:55,768][44958] Updated weights for policy 0, policy_version 44150 (0.0009) [2023-10-12 21:50:56,143][44958] Updated weights for policy 0, policy_version 44160 (0.0008) [2023-10-12 21:50:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90669056. Throughput: 0: 1638.2, 1: 1649.5. Samples: 22670092. Policy #0 lag: (min: 4.0, avg: 7.6, max: 36.0) [2023-10-12 21:50:56,444][43579] Avg episode reward: [(0, '278.080'), (1, '270.740')] [2023-10-12 21:50:59,595][44959] Updated weights for policy 1, policy_version 44390 (0.0008) [2023-10-12 21:50:59,963][44959] Updated weights for policy 1, policy_version 44400 (0.0007) [2023-10-12 21:51:00,331][44959] Updated weights for policy 1, policy_version 44410 (0.0007) [2023-10-12 21:51:00,349][44958] Updated weights for policy 0, policy_version 44170 (0.0007) [2023-10-12 21:51:00,730][44958] Updated weights for policy 0, policy_version 44180 (0.0009) [2023-10-12 21:51:01,111][44958] Updated weights for policy 0, policy_version 44190 (0.0008) [2023-10-12 21:51:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90734592. Throughput: 0: 1634.4, 1: 1647.4. Samples: 22688746. Policy #0 lag: (min: 4.0, avg: 7.6, max: 36.0) [2023-10-12 21:51:01,444][43579] Avg episode reward: [(0, '274.980'), (1, '277.060')] [2023-10-12 21:51:04,514][44959] Updated weights for policy 1, policy_version 44420 (0.0009) [2023-10-12 21:51:04,890][44959] Updated weights for policy 1, policy_version 44430 (0.0008) [2023-10-12 21:51:05,252][44959] Updated weights for policy 1, policy_version 44440 (0.0008) [2023-10-12 21:51:05,280][44958] Updated weights for policy 0, policy_version 44200 (0.0007) [2023-10-12 21:51:05,653][44958] Updated weights for policy 0, policy_version 44210 (0.0007) [2023-10-12 21:51:06,028][44958] Updated weights for policy 0, policy_version 44220 (0.0007) [2023-10-12 21:51:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90800128. Throughput: 0: 1640.2, 1: 1652.3. Samples: 22700102. Policy #0 lag: (min: 4.0, avg: 7.6, max: 36.0) [2023-10-12 21:51:06,444][43579] Avg episode reward: [(0, '272.660'), (1, '274.330')] [2023-10-12 21:51:09,430][44959] Updated weights for policy 1, policy_version 44450 (0.0009) [2023-10-12 21:51:09,796][44959] Updated weights for policy 1, policy_version 44460 (0.0007) [2023-10-12 21:51:10,170][44959] Updated weights for policy 1, policy_version 44470 (0.0009) [2023-10-12 21:51:10,389][44958] Updated weights for policy 0, policy_version 44230 (0.0009) [2023-10-12 21:51:10,547][44959] Updated weights for policy 1, policy_version 44480 (0.0009) [2023-10-12 21:51:10,774][44958] Updated weights for policy 0, policy_version 44240 (0.0008) [2023-10-12 21:51:11,140][44958] Updated weights for policy 0, policy_version 44250 (0.0010) [2023-10-12 21:51:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 90865664. Throughput: 0: 1637.6, 1: 1647.6. Samples: 22719332. Policy #0 lag: (min: 8.0, avg: 29.0, max: 40.0) [2023-10-12 21:51:11,443][43579] Avg episode reward: [(0, '271.710'), (1, '272.230')] [2023-10-12 21:51:14,795][44959] Updated weights for policy 1, policy_version 44490 (0.0009) [2023-10-12 21:51:15,160][44959] Updated weights for policy 1, policy_version 44500 (0.0008) [2023-10-12 21:51:15,268][44958] Updated weights for policy 0, policy_version 44260 (0.0010) [2023-10-12 21:51:15,529][44959] Updated weights for policy 1, policy_version 44510 (0.0008) [2023-10-12 21:51:15,649][44958] Updated weights for policy 0, policy_version 44270 (0.0010) [2023-10-12 21:51:16,019][44958] Updated weights for policy 0, policy_version 44280 (0.0010) [2023-10-12 21:51:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 90931200. Throughput: 0: 1636.1, 1: 1647.8. Samples: 22737740. Policy #0 lag: (min: 8.0, avg: 29.0, max: 40.0) [2023-10-12 21:51:16,443][43579] Avg episode reward: [(0, '271.220'), (1, '277.680')] [2023-10-12 21:51:19,691][44959] Updated weights for policy 1, policy_version 44520 (0.0010) [2023-10-12 21:51:20,067][44959] Updated weights for policy 1, policy_version 44530 (0.0008) [2023-10-12 21:51:20,160][44958] Updated weights for policy 0, policy_version 44290 (0.0008) [2023-10-12 21:51:20,434][44959] Updated weights for policy 1, policy_version 44540 (0.0007) [2023-10-12 21:51:20,540][44958] Updated weights for policy 0, policy_version 44300 (0.0008) [2023-10-12 21:51:20,911][44958] Updated weights for policy 0, policy_version 44310 (0.0007) [2023-10-12 21:51:21,291][44958] Updated weights for policy 0, policy_version 44320 (0.0008) [2023-10-12 21:51:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 90996736. Throughput: 0: 1630.7, 1: 1644.1. Samples: 22748712. Policy #0 lag: (min: 8.0, avg: 29.0, max: 40.0) [2023-10-12 21:51:21,444][43579] Avg episode reward: [(0, '267.590'), (1, '278.040')] [2023-10-12 21:51:24,416][44959] Updated weights for policy 1, policy_version 44550 (0.0007) [2023-10-12 21:51:24,785][44959] Updated weights for policy 1, policy_version 44560 (0.0007) [2023-10-12 21:51:25,152][44959] Updated weights for policy 1, policy_version 44570 (0.0009) [2023-10-12 21:51:25,545][44958] Updated weights for policy 0, policy_version 44330 (0.0009) [2023-10-12 21:51:25,910][44958] Updated weights for policy 0, policy_version 44340 (0.0010) [2023-10-12 21:51:26,292][44958] Updated weights for policy 0, policy_version 44350 (0.0009) [2023-10-12 21:51:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 91062272. Throughput: 0: 1629.7, 1: 1639.6. Samples: 22767934. Policy #0 lag: (min: 8.0, avg: 29.0, max: 40.0) [2023-10-12 21:51:26,444][43579] Avg episode reward: [(0, '273.780'), (1, '276.850')] [2023-10-12 21:51:29,500][44959] Updated weights for policy 1, policy_version 44580 (0.0009) [2023-10-12 21:51:29,864][44959] Updated weights for policy 1, policy_version 44590 (0.0010) [2023-10-12 21:51:30,238][44959] Updated weights for policy 1, policy_version 44600 (0.0007) [2023-10-12 21:51:30,306][44958] Updated weights for policy 0, policy_version 44360 (0.0010) [2023-10-12 21:51:30,674][44958] Updated weights for policy 0, policy_version 44370 (0.0009) [2023-10-12 21:51:31,044][44958] Updated weights for policy 0, policy_version 44380 (0.0007) [2023-10-12 21:51:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 91127808. Throughput: 0: 1628.8, 1: 1645.1. Samples: 22786720. Policy #0 lag: (min: 8.0, avg: 29.0, max: 40.0) [2023-10-12 21:51:31,443][43579] Avg episode reward: [(0, '276.970'), (1, '277.030')] [2023-10-12 21:51:34,496][44959] Updated weights for policy 1, policy_version 44610 (0.0009) [2023-10-12 21:51:34,863][44959] Updated weights for policy 1, policy_version 44620 (0.0009) [2023-10-12 21:51:35,234][44959] Updated weights for policy 1, policy_version 44630 (0.0009) [2023-10-12 21:51:35,377][44958] Updated weights for policy 0, policy_version 44390 (0.0009) [2023-10-12 21:51:35,610][44959] Updated weights for policy 1, policy_version 44640 (0.0008) [2023-10-12 21:51:35,735][44958] Updated weights for policy 0, policy_version 44400 (0.0008) [2023-10-12 21:51:36,114][44958] Updated weights for policy 0, policy_version 44410 (0.0007) [2023-10-12 21:51:36,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 91193344. Throughput: 0: 1627.4, 1: 1640.4. Samples: 22797650. Policy #0 lag: (min: 8.0, avg: 29.0, max: 40.0) [2023-10-12 21:51:36,444][43579] Avg episode reward: [(0, '271.070'), (1, '282.710')] [2023-10-12 21:51:39,717][44959] Updated weights for policy 1, policy_version 44650 (0.0007) [2023-10-12 21:51:40,082][44959] Updated weights for policy 1, policy_version 44660 (0.0008) [2023-10-12 21:51:40,406][44958] Updated weights for policy 0, policy_version 44420 (0.0008) [2023-10-12 21:51:40,449][44959] Updated weights for policy 1, policy_version 44670 (0.0008) [2023-10-12 21:51:40,777][44958] Updated weights for policy 0, policy_version 44430 (0.0007) [2023-10-12 21:51:41,148][44958] Updated weights for policy 0, policy_version 44440 (0.0010) [2023-10-12 21:51:41,443][43579] Fps is (10 sec: 9830.3, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 91226112. Throughput: 0: 1629.9, 1: 1635.4. Samples: 22817028. Policy #0 lag: (min: 8.0, avg: 29.0, max: 40.0) [2023-10-12 21:51:41,444][43579] Avg episode reward: [(0, '271.620'), (1, '280.870')] [2023-10-12 21:51:44,456][44959] Updated weights for policy 1, policy_version 44680 (0.0009) [2023-10-12 21:51:44,823][44959] Updated weights for policy 1, policy_version 44690 (0.0008) [2023-10-12 21:51:45,183][44959] Updated weights for policy 1, policy_version 44700 (0.0009) [2023-10-12 21:51:45,360][44958] Updated weights for policy 0, policy_version 44450 (0.0007) [2023-10-12 21:51:45,722][44958] Updated weights for policy 0, policy_version 44460 (0.0011) [2023-10-12 21:51:46,098][44958] Updated weights for policy 0, policy_version 44470 (0.0010) [2023-10-12 21:51:46,443][43579] Fps is (10 sec: 9830.5, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 91291648. Throughput: 0: 1632.0, 1: 1645.0. Samples: 22836212. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-12 21:51:46,443][43579] Avg episode reward: [(0, '274.300'), (1, '281.930')] [2023-10-12 21:51:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000044704_45776896.pth... [2023-10-12 21:51:46,463][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000044480_45547520.pth... [2023-10-12 21:51:46,470][44958] Updated weights for policy 0, policy_version 44480 (0.0008) [2023-10-12 21:51:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000043168_44204032.pth [2023-10-12 21:51:46,502][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000042944_43974656.pth [2023-10-12 21:51:49,604][44959] Updated weights for policy 1, policy_version 44710 (0.0008) [2023-10-12 21:51:49,970][44959] Updated weights for policy 1, policy_version 44720 (0.0009) [2023-10-12 21:51:50,325][44959] Updated weights for policy 1, policy_version 44730 (0.0007) [2023-10-12 21:51:50,745][44958] Updated weights for policy 0, policy_version 44490 (0.0008) [2023-10-12 21:51:51,115][44958] Updated weights for policy 0, policy_version 44500 (0.0010) [2023-10-12 21:51:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.0, 300 sec: 12996.1). Total num frames: 91357184. Throughput: 0: 1620.5, 1: 1639.9. Samples: 22846820. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-12 21:51:51,444][43579] Avg episode reward: [(0, '277.090'), (1, '282.200')] [2023-10-12 21:51:51,496][44958] Updated weights for policy 0, policy_version 44510 (0.0009) [2023-10-12 21:51:54,339][44959] Updated weights for policy 1, policy_version 44740 (0.0009) [2023-10-12 21:51:54,711][44959] Updated weights for policy 1, policy_version 44750 (0.0010) [2023-10-12 21:51:55,078][44959] Updated weights for policy 1, policy_version 44760 (0.0007) [2023-10-12 21:51:55,736][44958] Updated weights for policy 0, policy_version 44520 (0.0007) [2023-10-12 21:51:56,109][44958] Updated weights for policy 0, policy_version 44530 (0.0008) [2023-10-12 21:51:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 91422720. Throughput: 0: 1625.5, 1: 1640.3. Samples: 22866292. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-12 21:51:56,443][43579] Avg episode reward: [(0, '273.170'), (1, '278.040')] [2023-10-12 21:51:56,477][44958] Updated weights for policy 0, policy_version 44540 (0.0009) [2023-10-12 21:51:59,527][44959] Updated weights for policy 1, policy_version 44770 (0.0007) [2023-10-12 21:51:59,932][44959] Updated weights for policy 1, policy_version 44780 (0.0007) [2023-10-12 21:52:00,304][44959] Updated weights for policy 1, policy_version 44790 (0.0007) [2023-10-12 21:52:00,399][44958] Updated weights for policy 0, policy_version 44550 (0.0008) [2023-10-12 21:52:00,664][44959] Updated weights for policy 1, policy_version 44800 (0.0007) [2023-10-12 21:52:00,766][44958] Updated weights for policy 0, policy_version 44560 (0.0008) [2023-10-12 21:52:01,134][44958] Updated weights for policy 0, policy_version 44570 (0.0009) [2023-10-12 21:52:01,442][43579] Fps is (10 sec: 16384.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 91521024. Throughput: 0: 1637.8, 1: 1640.8. Samples: 22885280. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-12 21:52:01,443][43579] Avg episode reward: [(0, '273.170'), (1, '276.190')] [2023-10-12 21:52:04,715][44959] Updated weights for policy 1, policy_version 44810 (0.0007) [2023-10-12 21:52:05,070][44959] Updated weights for policy 1, policy_version 44820 (0.0009) [2023-10-12 21:52:05,266][44958] Updated weights for policy 0, policy_version 44580 (0.0009) [2023-10-12 21:52:05,439][44959] Updated weights for policy 1, policy_version 44830 (0.0008) [2023-10-12 21:52:05,640][44958] Updated weights for policy 0, policy_version 44590 (0.0008) [2023-10-12 21:52:06,013][44958] Updated weights for policy 0, policy_version 44600 (0.0008) [2023-10-12 21:52:06,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 91586560. Throughput: 0: 1636.6, 1: 1637.8. Samples: 22896060. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-12 21:52:06,443][43579] Avg episode reward: [(0, '273.220'), (1, '273.980')] [2023-10-12 21:52:09,671][44959] Updated weights for policy 1, policy_version 44840 (0.0010) [2023-10-12 21:52:10,031][44958] Updated weights for policy 0, policy_version 44610 (0.0010) [2023-10-12 21:52:10,034][44959] Updated weights for policy 1, policy_version 44850 (0.0009) [2023-10-12 21:52:10,395][44959] Updated weights for policy 1, policy_version 44860 (0.0009) [2023-10-12 21:52:10,413][44958] Updated weights for policy 0, policy_version 44620 (0.0007) [2023-10-12 21:52:10,789][44958] Updated weights for policy 0, policy_version 44630 (0.0007) [2023-10-12 21:52:11,164][44958] Updated weights for policy 0, policy_version 44640 (0.0007) [2023-10-12 21:52:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 91652096. Throughput: 0: 1637.8, 1: 1637.0. Samples: 22915298. Policy #0 lag: (min: 28.0, avg: 31.9, max: 60.0) [2023-10-12 21:52:11,444][43579] Avg episode reward: [(0, '270.240'), (1, '275.490')] [2023-10-12 21:52:14,784][44959] Updated weights for policy 1, policy_version 44870 (0.0008) [2023-10-12 21:52:15,156][44959] Updated weights for policy 1, policy_version 44880 (0.0009) [2023-10-12 21:52:15,353][44958] Updated weights for policy 0, policy_version 44650 (0.0008) [2023-10-12 21:52:15,522][44959] Updated weights for policy 1, policy_version 44890 (0.0008) [2023-10-12 21:52:15,717][44958] Updated weights for policy 0, policy_version 44660 (0.0008) [2023-10-12 21:52:16,084][44958] Updated weights for policy 0, policy_version 44670 (0.0010) [2023-10-12 21:52:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 91717632. Throughput: 0: 1638.1, 1: 1636.5. Samples: 22934080. Policy #0 lag: (min: 28.0, avg: 31.9, max: 60.0) [2023-10-12 21:52:16,444][43579] Avg episode reward: [(0, '266.910'), (1, '274.100')] [2023-10-12 21:52:19,675][44959] Updated weights for policy 1, policy_version 44900 (0.0008) [2023-10-12 21:52:20,045][44959] Updated weights for policy 1, policy_version 44910 (0.0007) [2023-10-12 21:52:20,409][44959] Updated weights for policy 1, policy_version 44920 (0.0007) [2023-10-12 21:52:20,485][44958] Updated weights for policy 0, policy_version 44680 (0.0009) [2023-10-12 21:52:20,851][44958] Updated weights for policy 0, policy_version 44690 (0.0008) [2023-10-12 21:52:21,225][44958] Updated weights for policy 0, policy_version 44700 (0.0009) [2023-10-12 21:52:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 91783168. Throughput: 0: 1639.0, 1: 1639.9. Samples: 22945198. Policy #0 lag: (min: 28.0, avg: 31.9, max: 60.0) [2023-10-12 21:52:21,443][43579] Avg episode reward: [(0, '268.640'), (1, '269.910')] [2023-10-12 21:52:24,618][44959] Updated weights for policy 1, policy_version 44930 (0.0008) [2023-10-12 21:52:24,990][44959] Updated weights for policy 1, policy_version 44940 (0.0008) [2023-10-12 21:52:25,270][44958] Updated weights for policy 0, policy_version 44710 (0.0009) [2023-10-12 21:52:25,353][44959] Updated weights for policy 1, policy_version 44950 (0.0007) [2023-10-12 21:52:25,639][44958] Updated weights for policy 0, policy_version 44720 (0.0008) [2023-10-12 21:52:25,717][44959] Updated weights for policy 1, policy_version 44960 (0.0007) [2023-10-12 21:52:26,017][44958] Updated weights for policy 0, policy_version 44730 (0.0007) [2023-10-12 21:52:26,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 91848704. Throughput: 0: 1643.8, 1: 1644.4. Samples: 22965000. Policy #0 lag: (min: 28.0, avg: 31.9, max: 60.0) [2023-10-12 21:52:26,444][43579] Avg episode reward: [(0, '270.840'), (1, '275.840')] [2023-10-12 21:52:29,835][44959] Updated weights for policy 1, policy_version 44970 (0.0007) [2023-10-12 21:52:30,120][44958] Updated weights for policy 0, policy_version 44740 (0.0008) [2023-10-12 21:52:30,203][44959] Updated weights for policy 1, policy_version 44980 (0.0008) [2023-10-12 21:52:30,498][44958] Updated weights for policy 0, policy_version 44750 (0.0009) [2023-10-12 21:52:30,572][44959] Updated weights for policy 1, policy_version 44990 (0.0008) [2023-10-12 21:52:30,862][44958] Updated weights for policy 0, policy_version 44760 (0.0010) [2023-10-12 21:52:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 91914240. Throughput: 0: 1638.7, 1: 1631.4. Samples: 22983368. Policy #0 lag: (min: 28.0, avg: 31.9, max: 60.0) [2023-10-12 21:52:31,443][43579] Avg episode reward: [(0, '270.480'), (1, '274.040')] [2023-10-12 21:52:34,846][44959] Updated weights for policy 1, policy_version 45000 (0.0008) [2023-10-12 21:52:34,975][44958] Updated weights for policy 0, policy_version 44770 (0.0008) [2023-10-12 21:52:35,218][44959] Updated weights for policy 1, policy_version 45010 (0.0007) [2023-10-12 21:52:35,352][44958] Updated weights for policy 0, policy_version 44780 (0.0008) [2023-10-12 21:52:35,581][44959] Updated weights for policy 1, policy_version 45020 (0.0009) [2023-10-12 21:52:35,717][44958] Updated weights for policy 0, policy_version 44790 (0.0007) [2023-10-12 21:52:36,092][44958] Updated weights for policy 0, policy_version 44800 (0.0008) [2023-10-12 21:52:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 91979776. Throughput: 0: 1651.5, 1: 1636.3. Samples: 22994770. Policy #0 lag: (min: 28.0, avg: 31.9, max: 60.0) [2023-10-12 21:52:36,443][43579] Avg episode reward: [(0, '266.440'), (1, '274.370')] [2023-10-12 21:52:39,672][44959] Updated weights for policy 1, policy_version 45030 (0.0009) [2023-10-12 21:52:40,043][44959] Updated weights for policy 1, policy_version 45040 (0.0009) [2023-10-12 21:52:40,198][44958] Updated weights for policy 0, policy_version 44810 (0.0009) [2023-10-12 21:52:40,411][44959] Updated weights for policy 1, policy_version 45050 (0.0007) [2023-10-12 21:52:40,574][44958] Updated weights for policy 0, policy_version 44820 (0.0009) [2023-10-12 21:52:40,940][44958] Updated weights for policy 0, policy_version 44830 (0.0009) [2023-10-12 21:52:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 92045312. Throughput: 0: 1646.1, 1: 1638.3. Samples: 23014090. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 21:52:41,444][43579] Avg episode reward: [(0, '270.900'), (1, '278.890')] [2023-10-12 21:52:44,560][44959] Updated weights for policy 1, policy_version 45060 (0.0008) [2023-10-12 21:52:44,937][44959] Updated weights for policy 1, policy_version 45070 (0.0008) [2023-10-12 21:52:45,305][44959] Updated weights for policy 1, policy_version 45080 (0.0009) [2023-10-12 21:52:45,345][44958] Updated weights for policy 0, policy_version 44840 (0.0009) [2023-10-12 21:52:45,718][44958] Updated weights for policy 0, policy_version 44850 (0.0010) [2023-10-12 21:52:46,101][44958] Updated weights for policy 0, policy_version 44860 (0.0007) [2023-10-12 21:52:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 92110848. Throughput: 0: 1639.8, 1: 1638.6. Samples: 23032808. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 21:52:46,443][43579] Avg episode reward: [(0, '269.050'), (1, '282.220')] [2023-10-12 21:52:49,544][44959] Updated weights for policy 1, policy_version 45090 (0.0007) [2023-10-12 21:52:49,917][44959] Updated weights for policy 1, policy_version 45100 (0.0007) [2023-10-12 21:52:50,255][44958] Updated weights for policy 0, policy_version 44870 (0.0008) [2023-10-12 21:52:50,281][44959] Updated weights for policy 1, policy_version 45110 (0.0011) [2023-10-12 21:52:50,625][44958] Updated weights for policy 0, policy_version 44880 (0.0009) [2023-10-12 21:52:50,654][44959] Updated weights for policy 1, policy_version 45120 (0.0010) [2023-10-12 21:52:51,004][44958] Updated weights for policy 0, policy_version 44890 (0.0009) [2023-10-12 21:52:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 92176384. Throughput: 0: 1642.1, 1: 1642.6. Samples: 23043872. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 21:52:51,444][43579] Avg episode reward: [(0, '266.520'), (1, '283.980')] [2023-10-12 21:52:54,749][44959] Updated weights for policy 1, policy_version 45130 (0.0008) [2023-10-12 21:52:55,058][44958] Updated weights for policy 0, policy_version 44900 (0.0008) [2023-10-12 21:52:55,122][44959] Updated weights for policy 1, policy_version 45140 (0.0008) [2023-10-12 21:52:55,442][44958] Updated weights for policy 0, policy_version 44910 (0.0009) [2023-10-12 21:52:55,481][44959] Updated weights for policy 1, policy_version 45150 (0.0009) [2023-10-12 21:52:55,810][44958] Updated weights for policy 0, policy_version 44920 (0.0008) [2023-10-12 21:52:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 92241920. Throughput: 0: 1642.9, 1: 1644.1. Samples: 23063214. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 21:52:56,444][43579] Avg episode reward: [(0, '271.110'), (1, '280.520')] [2023-10-12 21:52:59,715][44959] Updated weights for policy 1, policy_version 45160 (0.0009) [2023-10-12 21:52:59,930][44958] Updated weights for policy 0, policy_version 44930 (0.0007) [2023-10-12 21:53:00,074][44959] Updated weights for policy 1, policy_version 45170 (0.0008) [2023-10-12 21:53:00,308][44958] Updated weights for policy 0, policy_version 44940 (0.0008) [2023-10-12 21:53:00,447][44959] Updated weights for policy 1, policy_version 45180 (0.0009) [2023-10-12 21:53:00,685][44958] Updated weights for policy 0, policy_version 44950 (0.0009) [2023-10-12 21:53:01,062][44958] Updated weights for policy 0, policy_version 44960 (0.0010) [2023-10-12 21:53:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92307456. Throughput: 0: 1641.5, 1: 1643.2. Samples: 23081890. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 21:53:01,443][43579] Avg episode reward: [(0, '273.340'), (1, '284.460')] [2023-10-12 21:53:04,484][44959] Updated weights for policy 1, policy_version 45190 (0.0007) [2023-10-12 21:53:04,855][44959] Updated weights for policy 1, policy_version 45200 (0.0008) [2023-10-12 21:53:05,213][44959] Updated weights for policy 1, policy_version 45210 (0.0009) [2023-10-12 21:53:05,364][44958] Updated weights for policy 0, policy_version 44970 (0.0008) [2023-10-12 21:53:05,746][44958] Updated weights for policy 0, policy_version 44980 (0.0009) [2023-10-12 21:53:06,120][44958] Updated weights for policy 0, policy_version 44990 (0.0010) [2023-10-12 21:53:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 92372992. Throughput: 0: 1646.9, 1: 1644.5. Samples: 23093312. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 21:53:06,444][43579] Avg episode reward: [(0, '278.560'), (1, '285.020')] [2023-10-12 21:53:09,477][44959] Updated weights for policy 1, policy_version 45220 (0.0007) [2023-10-12 21:53:09,849][44959] Updated weights for policy 1, policy_version 45230 (0.0008) [2023-10-12 21:53:10,216][44959] Updated weights for policy 1, policy_version 45240 (0.0008) [2023-10-12 21:53:10,288][44958] Updated weights for policy 0, policy_version 45000 (0.0008) [2023-10-12 21:53:10,656][44958] Updated weights for policy 0, policy_version 45010 (0.0008) [2023-10-12 21:53:11,025][44958] Updated weights for policy 0, policy_version 45020 (0.0008) [2023-10-12 21:53:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92438528. Throughput: 0: 1647.0, 1: 1635.4. Samples: 23112708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:53:11,443][43579] Avg episode reward: [(0, '280.160'), (1, '284.860')] [2023-10-12 21:53:14,421][44959] Updated weights for policy 1, policy_version 45250 (0.0008) [2023-10-12 21:53:14,798][44959] Updated weights for policy 1, policy_version 45260 (0.0008) [2023-10-12 21:53:15,115][44958] Updated weights for policy 0, policy_version 45030 (0.0007) [2023-10-12 21:53:15,155][44959] Updated weights for policy 1, policy_version 45270 (0.0007) [2023-10-12 21:53:15,485][44958] Updated weights for policy 0, policy_version 45040 (0.0008) [2023-10-12 21:53:15,524][44959] Updated weights for policy 1, policy_version 45280 (0.0007) [2023-10-12 21:53:15,860][44958] Updated weights for policy 0, policy_version 45050 (0.0008) [2023-10-12 21:53:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92504064. Throughput: 0: 1645.2, 1: 1644.8. Samples: 23131418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:53:16,443][43579] Avg episode reward: [(0, '282.420'), (1, '285.180')] [2023-10-12 21:53:19,556][44959] Updated weights for policy 1, policy_version 45290 (0.0009) [2023-10-12 21:53:19,919][44959] Updated weights for policy 1, policy_version 45300 (0.0009) [2023-10-12 21:53:20,091][44958] Updated weights for policy 0, policy_version 45060 (0.0009) [2023-10-12 21:53:20,287][44959] Updated weights for policy 1, policy_version 45310 (0.0008) [2023-10-12 21:53:20,471][44958] Updated weights for policy 0, policy_version 45070 (0.0011) [2023-10-12 21:53:20,844][44958] Updated weights for policy 0, policy_version 45080 (0.0007) [2023-10-12 21:53:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92569600. Throughput: 0: 1644.6, 1: 1643.2. Samples: 23142720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:53:21,443][43579] Avg episode reward: [(0, '281.200'), (1, '287.150')] [2023-10-12 21:53:24,645][44959] Updated weights for policy 1, policy_version 45320 (0.0008) [2023-10-12 21:53:25,014][44958] Updated weights for policy 0, policy_version 45090 (0.0007) [2023-10-12 21:53:25,015][44959] Updated weights for policy 1, policy_version 45330 (0.0008) [2023-10-12 21:53:25,376][44958] Updated weights for policy 0, policy_version 45100 (0.0009) [2023-10-12 21:53:25,389][44959] Updated weights for policy 1, policy_version 45340 (0.0007) [2023-10-12 21:53:25,751][44958] Updated weights for policy 0, policy_version 45110 (0.0010) [2023-10-12 21:53:26,122][44958] Updated weights for policy 0, policy_version 45120 (0.0007) [2023-10-12 21:53:26,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92635136. Throughput: 0: 1642.0, 1: 1639.6. Samples: 23161762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:53:26,443][43579] Avg episode reward: [(0, '279.300'), (1, '288.210')] [2023-10-12 21:53:29,551][44959] Updated weights for policy 1, policy_version 45350 (0.0008) [2023-10-12 21:53:29,932][44959] Updated weights for policy 1, policy_version 45360 (0.0009) [2023-10-12 21:53:30,252][44958] Updated weights for policy 0, policy_version 45130 (0.0008) [2023-10-12 21:53:30,305][44959] Updated weights for policy 1, policy_version 45370 (0.0007) [2023-10-12 21:53:30,620][44958] Updated weights for policy 0, policy_version 45140 (0.0008) [2023-10-12 21:53:30,997][44958] Updated weights for policy 0, policy_version 45150 (0.0007) [2023-10-12 21:53:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92700672. Throughput: 0: 1639.6, 1: 1638.4. Samples: 23180316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:53:31,444][43579] Avg episode reward: [(0, '280.480'), (1, '286.180')] [2023-10-12 21:53:34,457][44959] Updated weights for policy 1, policy_version 45380 (0.0008) [2023-10-12 21:53:34,826][44959] Updated weights for policy 1, policy_version 45390 (0.0011) [2023-10-12 21:53:35,154][44958] Updated weights for policy 0, policy_version 45160 (0.0007) [2023-10-12 21:53:35,186][44959] Updated weights for policy 1, policy_version 45400 (0.0009) [2023-10-12 21:53:35,514][44958] Updated weights for policy 0, policy_version 45170 (0.0010) [2023-10-12 21:53:35,893][44958] Updated weights for policy 0, policy_version 45180 (0.0010) [2023-10-12 21:53:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92766208. Throughput: 0: 1646.9, 1: 1636.5. Samples: 23191624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:53:36,443][43579] Avg episode reward: [(0, '275.840'), (1, '280.820')] [2023-10-12 21:53:39,434][44959] Updated weights for policy 1, policy_version 45410 (0.0009) [2023-10-12 21:53:39,801][44959] Updated weights for policy 1, policy_version 45420 (0.0007) [2023-10-12 21:53:40,161][44959] Updated weights for policy 1, policy_version 45430 (0.0007) [2023-10-12 21:53:40,250][44958] Updated weights for policy 0, policy_version 45190 (0.0008) [2023-10-12 21:53:40,535][44959] Updated weights for policy 1, policy_version 45440 (0.0008) [2023-10-12 21:53:40,618][44958] Updated weights for policy 0, policy_version 45200 (0.0008) [2023-10-12 21:53:40,987][44958] Updated weights for policy 0, policy_version 45210 (0.0008) [2023-10-12 21:53:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92831744. Throughput: 0: 1643.3, 1: 1636.7. Samples: 23210816. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 21:53:41,443][43579] Avg episode reward: [(0, '274.890'), (1, '279.870')] [2023-10-12 21:53:44,655][44959] Updated weights for policy 1, policy_version 45450 (0.0008) [2023-10-12 21:53:45,022][44959] Updated weights for policy 1, policy_version 45460 (0.0010) [2023-10-12 21:53:45,049][44958] Updated weights for policy 0, policy_version 45220 (0.0009) [2023-10-12 21:53:45,396][44959] Updated weights for policy 1, policy_version 45470 (0.0008) [2023-10-12 21:53:45,422][44958] Updated weights for policy 0, policy_version 45230 (0.0007) [2023-10-12 21:53:45,789][44958] Updated weights for policy 0, policy_version 45240 (0.0007) [2023-10-12 21:53:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92897280. Throughput: 0: 1643.1, 1: 1638.1. Samples: 23229544. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 21:53:46,443][43579] Avg episode reward: [(0, '274.720'), (1, '276.030')] [2023-10-12 21:53:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000045472_46563328.pth... [2023-10-12 21:53:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000045248_46333952.pth... [2023-10-12 21:53:46,487][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000043712_44761088.pth [2023-10-12 21:53:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000043936_44990464.pth [2023-10-12 21:53:49,483][44959] Updated weights for policy 1, policy_version 45480 (0.0007) [2023-10-12 21:53:49,853][44959] Updated weights for policy 1, policy_version 45490 (0.0008) [2023-10-12 21:53:49,983][44958] Updated weights for policy 0, policy_version 45250 (0.0009) [2023-10-12 21:53:50,225][44959] Updated weights for policy 1, policy_version 45500 (0.0009) [2023-10-12 21:53:50,348][44958] Updated weights for policy 0, policy_version 45260 (0.0009) [2023-10-12 21:53:50,720][44958] Updated weights for policy 0, policy_version 45270 (0.0008) [2023-10-12 21:53:51,091][44958] Updated weights for policy 0, policy_version 45280 (0.0007) [2023-10-12 21:53:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 92962816. Throughput: 0: 1639.9, 1: 1638.1. Samples: 23240818. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 21:53:51,443][43579] Avg episode reward: [(0, '273.660'), (1, '271.250')] [2023-10-12 21:53:54,319][44959] Updated weights for policy 1, policy_version 45510 (0.0008) [2023-10-12 21:53:54,682][44959] Updated weights for policy 1, policy_version 45520 (0.0008) [2023-10-12 21:53:55,058][44959] Updated weights for policy 1, policy_version 45530 (0.0008) [2023-10-12 21:53:55,213][44958] Updated weights for policy 0, policy_version 45290 (0.0007) [2023-10-12 21:53:55,578][44958] Updated weights for policy 0, policy_version 45300 (0.0008) [2023-10-12 21:53:55,953][44958] Updated weights for policy 0, policy_version 45310 (0.0008) [2023-10-12 21:53:56,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93028352. Throughput: 0: 1632.9, 1: 1642.8. Samples: 23260118. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 21:53:56,444][43579] Avg episode reward: [(0, '273.960'), (1, '272.580')] [2023-10-12 21:53:59,284][44959] Updated weights for policy 1, policy_version 45540 (0.0007) [2023-10-12 21:53:59,654][44959] Updated weights for policy 1, policy_version 45550 (0.0007) [2023-10-12 21:54:00,015][44959] Updated weights for policy 1, policy_version 45560 (0.0008) [2023-10-12 21:54:00,060][44958] Updated weights for policy 0, policy_version 45320 (0.0008) [2023-10-12 21:54:00,443][44958] Updated weights for policy 0, policy_version 45330 (0.0008) [2023-10-12 21:54:00,818][44958] Updated weights for policy 0, policy_version 45340 (0.0009) [2023-10-12 21:54:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93093888. Throughput: 0: 1639.6, 1: 1640.6. Samples: 23279030. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 21:54:01,443][43579] Avg episode reward: [(0, '276.360'), (1, '276.060')] [2023-10-12 21:54:04,247][44959] Updated weights for policy 1, policy_version 45570 (0.0008) [2023-10-12 21:54:04,620][44959] Updated weights for policy 1, policy_version 45580 (0.0008) [2023-10-12 21:54:04,981][44959] Updated weights for policy 1, policy_version 45590 (0.0009) [2023-10-12 21:54:05,141][44958] Updated weights for policy 0, policy_version 45350 (0.0009) [2023-10-12 21:54:05,353][44959] Updated weights for policy 1, policy_version 45600 (0.0008) [2023-10-12 21:54:05,521][44958] Updated weights for policy 0, policy_version 45360 (0.0009) [2023-10-12 21:54:05,890][44958] Updated weights for policy 0, policy_version 45370 (0.0008) [2023-10-12 21:54:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93159424. Throughput: 0: 1640.7, 1: 1640.6. Samples: 23290378. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-12 21:54:06,443][43579] Avg episode reward: [(0, '280.080'), (1, '279.810')] [2023-10-12 21:54:09,484][44959] Updated weights for policy 1, policy_version 45610 (0.0009) [2023-10-12 21:54:09,852][44959] Updated weights for policy 1, policy_version 45620 (0.0007) [2023-10-12 21:54:10,031][44958] Updated weights for policy 0, policy_version 45380 (0.0009) [2023-10-12 21:54:10,219][44959] Updated weights for policy 1, policy_version 45630 (0.0008) [2023-10-12 21:54:10,398][44958] Updated weights for policy 0, policy_version 45390 (0.0010) [2023-10-12 21:54:10,769][44958] Updated weights for policy 0, policy_version 45400 (0.0009) [2023-10-12 21:54:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 93224960. Throughput: 0: 1644.3, 1: 1639.6. Samples: 23309538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:54:11,444][43579] Avg episode reward: [(0, '280.410'), (1, '275.060')] [2023-10-12 21:54:14,475][44959] Updated weights for policy 1, policy_version 45640 (0.0011) [2023-10-12 21:54:14,856][44959] Updated weights for policy 1, policy_version 45650 (0.0008) [2023-10-12 21:54:14,899][44958] Updated weights for policy 0, policy_version 45410 (0.0008) [2023-10-12 21:54:15,215][44959] Updated weights for policy 1, policy_version 45660 (0.0008) [2023-10-12 21:54:15,277][44958] Updated weights for policy 0, policy_version 45420 (0.0007) [2023-10-12 21:54:15,655][44958] Updated weights for policy 0, policy_version 45430 (0.0008) [2023-10-12 21:54:16,019][44958] Updated weights for policy 0, policy_version 45440 (0.0007) [2023-10-12 21:54:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93290496. Throughput: 0: 1647.7, 1: 1645.5. Samples: 23328506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:54:16,443][43579] Avg episode reward: [(0, '283.520'), (1, '278.300')] [2023-10-12 21:54:19,344][44959] Updated weights for policy 1, policy_version 45670 (0.0009) [2023-10-12 21:54:19,703][44959] Updated weights for policy 1, policy_version 45680 (0.0009) [2023-10-12 21:54:20,061][44959] Updated weights for policy 1, policy_version 45690 (0.0007) [2023-10-12 21:54:20,250][44958] Updated weights for policy 0, policy_version 45450 (0.0008) [2023-10-12 21:54:20,620][44958] Updated weights for policy 0, policy_version 45460 (0.0009) [2023-10-12 21:54:21,000][44958] Updated weights for policy 0, policy_version 45470 (0.0009) [2023-10-12 21:54:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 93356032. Throughput: 0: 1646.5, 1: 1646.3. Samples: 23339802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:54:21,444][43579] Avg episode reward: [(0, '281.330'), (1, '279.220')] [2023-10-12 21:54:24,298][44959] Updated weights for policy 1, policy_version 45700 (0.0008) [2023-10-12 21:54:24,665][44959] Updated weights for policy 1, policy_version 45710 (0.0009) [2023-10-12 21:54:25,023][44959] Updated weights for policy 1, policy_version 45720 (0.0008) [2023-10-12 21:54:25,057][44958] Updated weights for policy 0, policy_version 45480 (0.0009) [2023-10-12 21:54:25,432][44958] Updated weights for policy 0, policy_version 45490 (0.0008) [2023-10-12 21:54:25,792][44958] Updated weights for policy 0, policy_version 45500 (0.0008) [2023-10-12 21:54:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93421568. Throughput: 0: 1644.4, 1: 1644.5. Samples: 23358818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:54:26,443][43579] Avg episode reward: [(0, '277.540'), (1, '278.410')] [2023-10-12 21:54:29,192][44959] Updated weights for policy 1, policy_version 45730 (0.0009) [2023-10-12 21:54:29,566][44959] Updated weights for policy 1, policy_version 45740 (0.0008) [2023-10-12 21:54:29,841][44958] Updated weights for policy 0, policy_version 45510 (0.0007) [2023-10-12 21:54:29,931][44959] Updated weights for policy 1, policy_version 45750 (0.0009) [2023-10-12 21:54:30,211][44958] Updated weights for policy 0, policy_version 45520 (0.0008) [2023-10-12 21:54:30,295][44959] Updated weights for policy 1, policy_version 45760 (0.0007) [2023-10-12 21:54:30,579][44958] Updated weights for policy 0, policy_version 45530 (0.0007) [2023-10-12 21:54:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93487104. Throughput: 0: 1647.9, 1: 1650.9. Samples: 23377992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:54:31,443][43579] Avg episode reward: [(0, '266.540'), (1, '278.410')] [2023-10-12 21:54:34,482][44959] Updated weights for policy 1, policy_version 45770 (0.0008) [2023-10-12 21:54:34,854][44959] Updated weights for policy 1, policy_version 45780 (0.0007) [2023-10-12 21:54:34,872][44958] Updated weights for policy 0, policy_version 45540 (0.0008) [2023-10-12 21:54:35,228][44959] Updated weights for policy 1, policy_version 45790 (0.0007) [2023-10-12 21:54:35,242][44958] Updated weights for policy 0, policy_version 45550 (0.0008) [2023-10-12 21:54:35,615][44958] Updated weights for policy 0, policy_version 45560 (0.0009) [2023-10-12 21:54:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93552640. Throughput: 0: 1650.4, 1: 1649.3. Samples: 23389304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:54:36,443][43579] Avg episode reward: [(0, '263.510'), (1, '281.130')] [2023-10-12 21:54:39,168][44959] Updated weights for policy 1, policy_version 45800 (0.0010) [2023-10-12 21:54:39,533][44959] Updated weights for policy 1, policy_version 45810 (0.0007) [2023-10-12 21:54:39,744][44958] Updated weights for policy 0, policy_version 45570 (0.0009) [2023-10-12 21:54:39,897][44959] Updated weights for policy 1, policy_version 45820 (0.0008) [2023-10-12 21:54:40,111][44958] Updated weights for policy 0, policy_version 45580 (0.0009) [2023-10-12 21:54:40,479][44958] Updated weights for policy 0, policy_version 45590 (0.0009) [2023-10-12 21:54:40,844][44958] Updated weights for policy 0, policy_version 45600 (0.0009) [2023-10-12 21:54:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93618176. Throughput: 0: 1646.2, 1: 1641.7. Samples: 23408072. Policy #0 lag: (min: 22.0, avg: 23.2, max: 45.0) [2023-10-12 21:54:41,443][43579] Avg episode reward: [(0, '263.370'), (1, '284.360')] [2023-10-12 21:54:44,075][44959] Updated weights for policy 1, policy_version 45830 (0.0009) [2023-10-12 21:54:44,446][44959] Updated weights for policy 1, policy_version 45840 (0.0007) [2023-10-12 21:54:44,820][44959] Updated weights for policy 1, policy_version 45850 (0.0008) [2023-10-12 21:54:45,000][44958] Updated weights for policy 0, policy_version 45610 (0.0008) [2023-10-12 21:54:45,371][44958] Updated weights for policy 0, policy_version 45620 (0.0007) [2023-10-12 21:54:45,742][44958] Updated weights for policy 0, policy_version 45630 (0.0009) [2023-10-12 21:54:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93683712. Throughput: 0: 1646.6, 1: 1655.5. Samples: 23427622. Policy #0 lag: (min: 22.0, avg: 23.2, max: 45.0) [2023-10-12 21:54:46,443][43579] Avg episode reward: [(0, '268.210'), (1, '285.890')] [2023-10-12 21:54:48,594][44959] Updated weights for policy 1, policy_version 45860 (0.0008) [2023-10-12 21:54:48,961][44959] Updated weights for policy 1, policy_version 45870 (0.0008) [2023-10-12 21:54:49,330][44959] Updated weights for policy 1, policy_version 45880 (0.0009) [2023-10-12 21:54:49,881][44958] Updated weights for policy 0, policy_version 45640 (0.0009) [2023-10-12 21:54:50,259][44958] Updated weights for policy 0, policy_version 45650 (0.0010) [2023-10-12 21:54:50,636][44958] Updated weights for policy 0, policy_version 45660 (0.0008) [2023-10-12 21:54:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93749248. Throughput: 0: 1646.4, 1: 1641.7. Samples: 23438346. Policy #0 lag: (min: 22.0, avg: 23.2, max: 45.0) [2023-10-12 21:54:51,443][43579] Avg episode reward: [(0, '266.070'), (1, '283.910')] [2023-10-12 21:54:53,614][44959] Updated weights for policy 1, policy_version 45890 (0.0007) [2023-10-12 21:54:53,988][44959] Updated weights for policy 1, policy_version 45900 (0.0007) [2023-10-12 21:54:54,357][44959] Updated weights for policy 1, policy_version 45910 (0.0007) [2023-10-12 21:54:54,722][44959] Updated weights for policy 1, policy_version 45920 (0.0007) [2023-10-12 21:54:54,832][44958] Updated weights for policy 0, policy_version 45670 (0.0009) [2023-10-12 21:54:55,202][44958] Updated weights for policy 0, policy_version 45680 (0.0007) [2023-10-12 21:54:55,570][44958] Updated weights for policy 0, policy_version 45690 (0.0008) [2023-10-12 21:54:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93814784. Throughput: 0: 1638.6, 1: 1643.6. Samples: 23457236. Policy #0 lag: (min: 22.0, avg: 23.2, max: 45.0) [2023-10-12 21:54:56,444][43579] Avg episode reward: [(0, '272.740'), (1, '284.720')] [2023-10-12 21:54:59,050][44959] Updated weights for policy 1, policy_version 45930 (0.0010) [2023-10-12 21:54:59,429][44959] Updated weights for policy 1, policy_version 45940 (0.0009) [2023-10-12 21:54:59,787][44958] Updated weights for policy 0, policy_version 45700 (0.0009) [2023-10-12 21:54:59,799][44959] Updated weights for policy 1, policy_version 45950 (0.0009) [2023-10-12 21:55:00,168][44958] Updated weights for policy 0, policy_version 45710 (0.0011) [2023-10-12 21:55:00,545][44958] Updated weights for policy 0, policy_version 45720 (0.0009) [2023-10-12 21:55:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93880320. Throughput: 0: 1638.8, 1: 1651.3. Samples: 23476558. Policy #0 lag: (min: 22.0, avg: 23.2, max: 45.0) [2023-10-12 21:55:01,443][43579] Avg episode reward: [(0, '280.140'), (1, '277.130')] [2023-10-12 21:55:04,052][44959] Updated weights for policy 1, policy_version 45960 (0.0008) [2023-10-12 21:55:04,438][44959] Updated weights for policy 1, policy_version 45970 (0.0009) [2023-10-12 21:55:04,682][44958] Updated weights for policy 0, policy_version 45730 (0.0008) [2023-10-12 21:55:04,807][44959] Updated weights for policy 1, policy_version 45980 (0.0009) [2023-10-12 21:55:05,041][44958] Updated weights for policy 0, policy_version 45740 (0.0009) [2023-10-12 21:55:05,411][44958] Updated weights for policy 0, policy_version 45750 (0.0010) [2023-10-12 21:55:05,783][44958] Updated weights for policy 0, policy_version 45760 (0.0007) [2023-10-12 21:55:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 93945856. Throughput: 0: 1638.9, 1: 1643.8. Samples: 23487526. Policy #0 lag: (min: 22.0, avg: 23.2, max: 45.0) [2023-10-12 21:55:06,444][43579] Avg episode reward: [(0, '282.300'), (1, '272.390')] [2023-10-12 21:55:08,934][44959] Updated weights for policy 1, policy_version 45990 (0.0007) [2023-10-12 21:55:09,299][44959] Updated weights for policy 1, policy_version 46000 (0.0008) [2023-10-12 21:55:09,674][44959] Updated weights for policy 1, policy_version 46010 (0.0009) [2023-10-12 21:55:10,147][44958] Updated weights for policy 0, policy_version 45770 (0.0007) [2023-10-12 21:55:10,519][44958] Updated weights for policy 0, policy_version 45780 (0.0008) [2023-10-12 21:55:10,894][44958] Updated weights for policy 0, policy_version 45790 (0.0009) [2023-10-12 21:55:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 94011392. Throughput: 0: 1634.4, 1: 1642.2. Samples: 23506266. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:55:11,443][43579] Avg episode reward: [(0, '281.670'), (1, '272.660')] [2023-10-12 21:55:13,859][44959] Updated weights for policy 1, policy_version 46020 (0.0009) [2023-10-12 21:55:14,235][44959] Updated weights for policy 1, policy_version 46030 (0.0008) [2023-10-12 21:55:14,604][44959] Updated weights for policy 1, policy_version 46040 (0.0008) [2023-10-12 21:55:14,910][44958] Updated weights for policy 0, policy_version 45800 (0.0008) [2023-10-12 21:55:15,284][44958] Updated weights for policy 0, policy_version 45810 (0.0007) [2023-10-12 21:55:15,644][44958] Updated weights for policy 0, policy_version 45820 (0.0008) [2023-10-12 21:55:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94076928. Throughput: 0: 1637.9, 1: 1646.5. Samples: 23525792. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:55:16,443][43579] Avg episode reward: [(0, '280.420'), (1, '276.270')] [2023-10-12 21:55:18,766][44959] Updated weights for policy 1, policy_version 46050 (0.0007) [2023-10-12 21:55:19,129][44959] Updated weights for policy 1, policy_version 46060 (0.0007) [2023-10-12 21:55:19,495][44959] Updated weights for policy 1, policy_version 46070 (0.0008) [2023-10-12 21:55:19,860][44959] Updated weights for policy 1, policy_version 46080 (0.0010) [2023-10-12 21:55:20,149][44958] Updated weights for policy 0, policy_version 45830 (0.0008) [2023-10-12 21:55:20,512][44958] Updated weights for policy 0, policy_version 45840 (0.0008) [2023-10-12 21:55:20,886][44958] Updated weights for policy 0, policy_version 45850 (0.0009) [2023-10-12 21:55:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 94142464. Throughput: 0: 1634.6, 1: 1638.8. Samples: 23536610. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:55:21,443][43579] Avg episode reward: [(0, '282.590'), (1, '276.930')] [2023-10-12 21:55:24,049][44959] Updated weights for policy 1, policy_version 46090 (0.0009) [2023-10-12 21:55:24,414][44959] Updated weights for policy 1, policy_version 46100 (0.0009) [2023-10-12 21:55:24,781][44959] Updated weights for policy 1, policy_version 46110 (0.0009) [2023-10-12 21:55:24,999][44958] Updated weights for policy 0, policy_version 45860 (0.0008) [2023-10-12 21:55:25,373][44958] Updated weights for policy 0, policy_version 45870 (0.0007) [2023-10-12 21:55:25,743][44958] Updated weights for policy 0, policy_version 45880 (0.0009) [2023-10-12 21:55:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94208000. Throughput: 0: 1641.6, 1: 1642.3. Samples: 23555852. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:55:26,444][43579] Avg episode reward: [(0, '277.350'), (1, '280.060')] [2023-10-12 21:55:28,972][44959] Updated weights for policy 1, policy_version 46120 (0.0008) [2023-10-12 21:55:29,341][44959] Updated weights for policy 1, policy_version 46130 (0.0007) [2023-10-12 21:55:29,707][44959] Updated weights for policy 1, policy_version 46140 (0.0007) [2023-10-12 21:55:29,978][44958] Updated weights for policy 0, policy_version 45890 (0.0007) [2023-10-12 21:55:30,353][44958] Updated weights for policy 0, policy_version 45900 (0.0007) [2023-10-12 21:55:30,725][44958] Updated weights for policy 0, policy_version 45910 (0.0008) [2023-10-12 21:55:31,100][44958] Updated weights for policy 0, policy_version 45920 (0.0007) [2023-10-12 21:55:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94273536. Throughput: 0: 1637.1, 1: 1639.9. Samples: 23575088. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:55:31,443][43579] Avg episode reward: [(0, '277.240'), (1, '285.030')] [2023-10-12 21:55:33,981][44959] Updated weights for policy 1, policy_version 46150 (0.0008) [2023-10-12 21:55:34,346][44959] Updated weights for policy 1, policy_version 46160 (0.0009) [2023-10-12 21:55:34,727][44959] Updated weights for policy 1, policy_version 46170 (0.0008) [2023-10-12 21:55:35,349][44958] Updated weights for policy 0, policy_version 45930 (0.0009) [2023-10-12 21:55:35,731][44958] Updated weights for policy 0, policy_version 45940 (0.0009) [2023-10-12 21:55:36,100][44958] Updated weights for policy 0, policy_version 45950 (0.0011) [2023-10-12 21:55:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94339072. Throughput: 0: 1636.0, 1: 1647.3. Samples: 23586096. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 21:55:36,443][43579] Avg episode reward: [(0, '275.500'), (1, '288.950')] [2023-10-12 21:55:38,928][44959] Updated weights for policy 1, policy_version 46180 (0.0009) [2023-10-12 21:55:39,296][44959] Updated weights for policy 1, policy_version 46190 (0.0007) [2023-10-12 21:55:39,665][44959] Updated weights for policy 1, policy_version 46200 (0.0011) [2023-10-12 21:55:40,240][44958] Updated weights for policy 0, policy_version 45960 (0.0011) [2023-10-12 21:55:40,626][44958] Updated weights for policy 0, policy_version 45970 (0.0008) [2023-10-12 21:55:40,989][44958] Updated weights for policy 0, policy_version 45980 (0.0007) [2023-10-12 21:55:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94404608. Throughput: 0: 1642.6, 1: 1640.5. Samples: 23604978. Policy #0 lag: (min: 1.0, avg: 21.5, max: 33.0) [2023-10-12 21:55:41,443][43579] Avg episode reward: [(0, '279.970'), (1, '287.320')] [2023-10-12 21:55:43,691][44959] Updated weights for policy 1, policy_version 46210 (0.0009) [2023-10-12 21:55:44,062][44959] Updated weights for policy 1, policy_version 46220 (0.0008) [2023-10-12 21:55:44,430][44959] Updated weights for policy 1, policy_version 46230 (0.0007) [2023-10-12 21:55:44,798][44959] Updated weights for policy 1, policy_version 46240 (0.0007) [2023-10-12 21:55:45,177][44958] Updated weights for policy 0, policy_version 45990 (0.0008) [2023-10-12 21:55:45,547][44958] Updated weights for policy 0, policy_version 46000 (0.0008) [2023-10-12 21:55:45,929][44958] Updated weights for policy 0, policy_version 46010 (0.0009) [2023-10-12 21:55:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94470144. Throughput: 0: 1638.2, 1: 1645.2. Samples: 23624310. Policy #0 lag: (min: 1.0, avg: 21.5, max: 33.0) [2023-10-12 21:55:46,443][43579] Avg episode reward: [(0, '271.400'), (1, '283.390')] [2023-10-12 21:55:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000046240_47349760.pth... [2023-10-12 21:55:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000046016_47120384.pth... [2023-10-12 21:55:46,485][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000044704_45776896.pth [2023-10-12 21:55:46,489][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000046240_47349760.pth [2023-10-12 21:55:46,490][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000044480_45547520.pth [2023-10-12 21:55:46,494][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000046016_47120384.pth [2023-10-12 21:55:49,123][44959] Updated weights for policy 1, policy_version 46250 (0.0008) [2023-10-12 21:55:49,494][44959] Updated weights for policy 1, policy_version 46260 (0.0008) [2023-10-12 21:55:49,861][44959] Updated weights for policy 1, policy_version 46270 (0.0009) [2023-10-12 21:55:50,028][44958] Updated weights for policy 0, policy_version 46020 (0.0009) [2023-10-12 21:55:50,399][44958] Updated weights for policy 0, policy_version 46030 (0.0007) [2023-10-12 21:55:50,758][44958] Updated weights for policy 0, policy_version 46040 (0.0008) [2023-10-12 21:55:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94535680. Throughput: 0: 1635.6, 1: 1645.9. Samples: 23635190. Policy #0 lag: (min: 1.0, avg: 21.5, max: 33.0) [2023-10-12 21:55:51,443][43579] Avg episode reward: [(0, '275.220'), (1, '284.650')] [2023-10-12 21:55:54,102][44959] Updated weights for policy 1, policy_version 46280 (0.0007) [2023-10-12 21:55:54,463][44959] Updated weights for policy 1, policy_version 46290 (0.0009) [2023-10-12 21:55:54,782][44958] Updated weights for policy 0, policy_version 46050 (0.0007) [2023-10-12 21:55:54,831][44959] Updated weights for policy 1, policy_version 46300 (0.0009) [2023-10-12 21:55:55,157][44958] Updated weights for policy 0, policy_version 46060 (0.0007) [2023-10-12 21:55:55,536][44958] Updated weights for policy 0, policy_version 46070 (0.0008) [2023-10-12 21:55:55,899][44958] Updated weights for policy 0, policy_version 46080 (0.0008) [2023-10-12 21:55:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94601216. Throughput: 0: 1642.7, 1: 1642.5. Samples: 23654098. Policy #0 lag: (min: 1.0, avg: 21.5, max: 33.0) [2023-10-12 21:55:56,444][43579] Avg episode reward: [(0, '272.340'), (1, '282.290')] [2023-10-12 21:55:58,968][44959] Updated weights for policy 1, policy_version 46310 (0.0007) [2023-10-12 21:55:59,334][44959] Updated weights for policy 1, policy_version 46320 (0.0009) [2023-10-12 21:55:59,703][44959] Updated weights for policy 1, policy_version 46330 (0.0009) [2023-10-12 21:56:00,018][44958] Updated weights for policy 0, policy_version 46090 (0.0008) [2023-10-12 21:56:00,396][44958] Updated weights for policy 0, policy_version 46100 (0.0010) [2023-10-12 21:56:00,767][44958] Updated weights for policy 0, policy_version 46110 (0.0010) [2023-10-12 21:56:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 94666752. Throughput: 0: 1641.9, 1: 1642.0. Samples: 23673568. Policy #0 lag: (min: 1.0, avg: 21.5, max: 33.0) [2023-10-12 21:56:01,444][43579] Avg episode reward: [(0, '277.040'), (1, '278.680')] [2023-10-12 21:56:03,920][44959] Updated weights for policy 1, policy_version 46340 (0.0009) [2023-10-12 21:56:04,283][44959] Updated weights for policy 1, policy_version 46350 (0.0010) [2023-10-12 21:56:04,660][44959] Updated weights for policy 1, policy_version 46360 (0.0011) [2023-10-12 21:56:05,063][44958] Updated weights for policy 0, policy_version 46120 (0.0008) [2023-10-12 21:56:05,438][44958] Updated weights for policy 0, policy_version 46130 (0.0009) [2023-10-12 21:56:05,808][44958] Updated weights for policy 0, policy_version 46140 (0.0008) [2023-10-12 21:56:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94732288. Throughput: 0: 1643.6, 1: 1643.5. Samples: 23684530. Policy #0 lag: (min: 1.0, avg: 21.5, max: 33.0) [2023-10-12 21:56:06,443][43579] Avg episode reward: [(0, '273.160'), (1, '279.970')] [2023-10-12 21:56:08,671][44959] Updated weights for policy 1, policy_version 46370 (0.0007) [2023-10-12 21:56:09,032][44959] Updated weights for policy 1, policy_version 46380 (0.0007) [2023-10-12 21:56:09,403][44959] Updated weights for policy 1, policy_version 46390 (0.0007) [2023-10-12 21:56:09,774][44959] Updated weights for policy 1, policy_version 46400 (0.0008) [2023-10-12 21:56:09,902][44958] Updated weights for policy 0, policy_version 46150 (0.0008) [2023-10-12 21:56:10,283][44958] Updated weights for policy 0, policy_version 46160 (0.0009) [2023-10-12 21:56:10,657][44958] Updated weights for policy 0, policy_version 46170 (0.0008) [2023-10-12 21:56:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94797824. Throughput: 0: 1635.3, 1: 1641.8. Samples: 23703320. Policy #0 lag: (min: 0.0, avg: 22.7, max: 32.0) [2023-10-12 21:56:11,443][43579] Avg episode reward: [(0, '274.740'), (1, '279.060')] [2023-10-12 21:56:13,924][44959] Updated weights for policy 1, policy_version 46410 (0.0009) [2023-10-12 21:56:14,291][44959] Updated weights for policy 1, policy_version 46420 (0.0008) [2023-10-12 21:56:14,664][44959] Updated weights for policy 1, policy_version 46430 (0.0009) [2023-10-12 21:56:14,788][44958] Updated weights for policy 0, policy_version 46180 (0.0009) [2023-10-12 21:56:15,168][44958] Updated weights for policy 0, policy_version 46190 (0.0010) [2023-10-12 21:56:15,537][44958] Updated weights for policy 0, policy_version 46200 (0.0009) [2023-10-12 21:56:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94863360. Throughput: 0: 1638.2, 1: 1643.3. Samples: 23722756. Policy #0 lag: (min: 0.0, avg: 22.7, max: 32.0) [2023-10-12 21:56:16,443][43579] Avg episode reward: [(0, '280.330'), (1, '283.410')] [2023-10-12 21:56:18,862][44959] Updated weights for policy 1, policy_version 46440 (0.0009) [2023-10-12 21:56:19,225][44959] Updated weights for policy 1, policy_version 46450 (0.0009) [2023-10-12 21:56:19,590][44959] Updated weights for policy 1, policy_version 46460 (0.0007) [2023-10-12 21:56:19,734][44958] Updated weights for policy 0, policy_version 46210 (0.0008) [2023-10-12 21:56:20,103][44958] Updated weights for policy 0, policy_version 46220 (0.0010) [2023-10-12 21:56:20,482][44958] Updated weights for policy 0, policy_version 46230 (0.0008) [2023-10-12 21:56:20,852][44958] Updated weights for policy 0, policy_version 46240 (0.0009) [2023-10-12 21:56:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94928896. Throughput: 0: 1638.0, 1: 1640.0. Samples: 23733606. Policy #0 lag: (min: 0.0, avg: 22.7, max: 32.0) [2023-10-12 21:56:21,443][43579] Avg episode reward: [(0, '283.200'), (1, '281.140')] [2023-10-12 21:56:23,576][44959] Updated weights for policy 1, policy_version 46470 (0.0009) [2023-10-12 21:56:23,943][44959] Updated weights for policy 1, policy_version 46480 (0.0009) [2023-10-12 21:56:24,322][44959] Updated weights for policy 1, policy_version 46490 (0.0007) [2023-10-12 21:56:25,062][44958] Updated weights for policy 0, policy_version 46250 (0.0009) [2023-10-12 21:56:25,438][44958] Updated weights for policy 0, policy_version 46260 (0.0007) [2023-10-12 21:56:25,808][44958] Updated weights for policy 0, policy_version 46270 (0.0007) [2023-10-12 21:56:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 94994432. Throughput: 0: 1635.0, 1: 1653.9. Samples: 23752980. Policy #0 lag: (min: 0.0, avg: 22.7, max: 32.0) [2023-10-12 21:56:26,444][43579] Avg episode reward: [(0, '281.160'), (1, '280.840')] [2023-10-12 21:56:28,408][44959] Updated weights for policy 1, policy_version 46500 (0.0008) [2023-10-12 21:56:28,771][44959] Updated weights for policy 1, policy_version 46510 (0.0008) [2023-10-12 21:56:29,146][44959] Updated weights for policy 1, policy_version 46520 (0.0010) [2023-10-12 21:56:30,016][44958] Updated weights for policy 0, policy_version 46280 (0.0008) [2023-10-12 21:56:30,394][44958] Updated weights for policy 0, policy_version 46290 (0.0009) [2023-10-12 21:56:30,758][44958] Updated weights for policy 0, policy_version 46300 (0.0008) [2023-10-12 21:56:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 95059968. Throughput: 0: 1637.1, 1: 1651.2. Samples: 23772286. Policy #0 lag: (min: 0.0, avg: 22.7, max: 32.0) [2023-10-12 21:56:31,443][43579] Avg episode reward: [(0, '277.040'), (1, '283.220')] [2023-10-12 21:56:33,358][44959] Updated weights for policy 1, policy_version 46530 (0.0009) [2023-10-12 21:56:33,725][44959] Updated weights for policy 1, policy_version 46540 (0.0008) [2023-10-12 21:56:34,094][44959] Updated weights for policy 1, policy_version 46550 (0.0008) [2023-10-12 21:56:34,461][44959] Updated weights for policy 1, policy_version 46560 (0.0007) [2023-10-12 21:56:34,805][44958] Updated weights for policy 0, policy_version 46310 (0.0008) [2023-10-12 21:56:35,182][44958] Updated weights for policy 0, policy_version 46320 (0.0010) [2023-10-12 21:56:35,558][44958] Updated weights for policy 0, policy_version 46330 (0.0009) [2023-10-12 21:56:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 95125504. Throughput: 0: 1641.2, 1: 1644.5. Samples: 23783048. Policy #0 lag: (min: 0.0, avg: 22.7, max: 32.0) [2023-10-12 21:56:36,444][43579] Avg episode reward: [(0, '276.310'), (1, '280.600')] [2023-10-12 21:56:38,393][44959] Updated weights for policy 1, policy_version 46570 (0.0007) [2023-10-12 21:56:38,752][44959] Updated weights for policy 1, policy_version 46580 (0.0008) [2023-10-12 21:56:39,123][44959] Updated weights for policy 1, policy_version 46590 (0.0009) [2023-10-12 21:56:39,846][44958] Updated weights for policy 0, policy_version 46340 (0.0009) [2023-10-12 21:56:40,214][44958] Updated weights for policy 0, policy_version 46350 (0.0010) [2023-10-12 21:56:40,585][44958] Updated weights for policy 0, policy_version 46360 (0.0009) [2023-10-12 21:56:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 95191040. Throughput: 0: 1634.0, 1: 1664.5. Samples: 23802532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:56:41,444][43579] Avg episode reward: [(0, '274.920'), (1, '281.410')] [2023-10-12 21:56:43,153][44959] Updated weights for policy 1, policy_version 46600 (0.0008) [2023-10-12 21:56:43,521][44959] Updated weights for policy 1, policy_version 46610 (0.0009) [2023-10-12 21:56:43,893][44959] Updated weights for policy 1, policy_version 46620 (0.0010) [2023-10-12 21:56:44,552][44958] Updated weights for policy 0, policy_version 46370 (0.0008) [2023-10-12 21:56:44,929][44958] Updated weights for policy 0, policy_version 46380 (0.0008) [2023-10-12 21:56:45,306][44958] Updated weights for policy 0, policy_version 46390 (0.0008) [2023-10-12 21:56:45,669][44958] Updated weights for policy 0, policy_version 46400 (0.0008) [2023-10-12 21:56:46,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 95256576. Throughput: 0: 1636.9, 1: 1670.2. Samples: 23822388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:56:46,443][43579] Avg episode reward: [(0, '270.300'), (1, '280.260')] [2023-10-12 21:56:48,008][44959] Updated weights for policy 1, policy_version 46630 (0.0010) [2023-10-12 21:56:48,382][44959] Updated weights for policy 1, policy_version 46640 (0.0008) [2023-10-12 21:56:48,749][44959] Updated weights for policy 1, policy_version 46650 (0.0010) [2023-10-12 21:56:49,886][44958] Updated weights for policy 0, policy_version 46410 (0.0010) [2023-10-12 21:56:50,260][44958] Updated weights for policy 0, policy_version 46420 (0.0010) [2023-10-12 21:56:50,634][44958] Updated weights for policy 0, policy_version 46430 (0.0011) [2023-10-12 21:56:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 95322112. Throughput: 0: 1640.6, 1: 1650.0. Samples: 23832608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:56:51,443][43579] Avg episode reward: [(0, '272.410'), (1, '281.440')] [2023-10-12 21:56:53,054][44959] Updated weights for policy 1, policy_version 46660 (0.0009) [2023-10-12 21:56:53,425][44959] Updated weights for policy 1, policy_version 46670 (0.0008) [2023-10-12 21:56:53,790][44959] Updated weights for policy 1, policy_version 46680 (0.0008) [2023-10-12 21:56:54,722][44958] Updated weights for policy 0, policy_version 46440 (0.0009) [2023-10-12 21:56:55,098][44958] Updated weights for policy 0, policy_version 46450 (0.0010) [2023-10-12 21:56:55,456][44958] Updated weights for policy 0, policy_version 46460 (0.0010) [2023-10-12 21:56:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 95387648. Throughput: 0: 1632.6, 1: 1674.4. Samples: 23852136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:56:56,443][43579] Avg episode reward: [(0, '273.600'), (1, '279.860')] [2023-10-12 21:56:57,839][44959] Updated weights for policy 1, policy_version 46690 (0.0010) [2023-10-12 21:56:58,208][44959] Updated weights for policy 1, policy_version 46700 (0.0010) [2023-10-12 21:56:58,570][44959] Updated weights for policy 1, policy_version 46710 (0.0011) [2023-10-12 21:56:58,940][44959] Updated weights for policy 1, policy_version 46720 (0.0008) [2023-10-12 21:56:59,797][44958] Updated weights for policy 0, policy_version 46470 (0.0008) [2023-10-12 21:57:00,163][44958] Updated weights for policy 0, policy_version 46480 (0.0008) [2023-10-12 21:57:00,533][44958] Updated weights for policy 0, policy_version 46490 (0.0009) [2023-10-12 21:57:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 95453184. Throughput: 0: 1635.6, 1: 1672.9. Samples: 23871640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:01,444][43579] Avg episode reward: [(0, '269.620'), (1, '280.150')] [2023-10-12 21:57:03,187][44959] Updated weights for policy 1, policy_version 46730 (0.0009) [2023-10-12 21:57:03,559][44959] Updated weights for policy 1, policy_version 46740 (0.0009) [2023-10-12 21:57:03,928][44959] Updated weights for policy 1, policy_version 46750 (0.0008) [2023-10-12 21:57:04,580][44958] Updated weights for policy 0, policy_version 46500 (0.0010) [2023-10-12 21:57:04,954][44958] Updated weights for policy 0, policy_version 46510 (0.0009) [2023-10-12 21:57:05,332][44958] Updated weights for policy 0, policy_version 46520 (0.0011) [2023-10-12 21:57:06,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 95518720. Throughput: 0: 1639.6, 1: 1654.5. Samples: 23881842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:06,444][43579] Avg episode reward: [(0, '272.280'), (1, '276.730')] [2023-10-12 21:57:07,903][44959] Updated weights for policy 1, policy_version 46760 (0.0007) [2023-10-12 21:57:08,265][44959] Updated weights for policy 1, policy_version 46770 (0.0010) [2023-10-12 21:57:08,628][44959] Updated weights for policy 1, policy_version 46780 (0.0010) [2023-10-12 21:57:09,585][44958] Updated weights for policy 0, policy_version 46530 (0.0011) [2023-10-12 21:57:09,965][44958] Updated weights for policy 0, policy_version 46540 (0.0007) [2023-10-12 21:57:10,338][44958] Updated weights for policy 0, policy_version 46550 (0.0007) [2023-10-12 21:57:10,701][44958] Updated weights for policy 0, policy_version 46560 (0.0007) [2023-10-12 21:57:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 95584256. Throughput: 0: 1632.0, 1: 1667.7. Samples: 23901466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:11,444][43579] Avg episode reward: [(0, '273.080'), (1, '276.590')] [2023-10-12 21:57:12,962][44959] Updated weights for policy 1, policy_version 46790 (0.0008) [2023-10-12 21:57:13,320][44959] Updated weights for policy 1, policy_version 46800 (0.0008) [2023-10-12 21:57:13,693][44959] Updated weights for policy 1, policy_version 46810 (0.0009) [2023-10-12 21:57:15,077][44958] Updated weights for policy 0, policy_version 46570 (0.0010) [2023-10-12 21:57:15,451][44958] Updated weights for policy 0, policy_version 46580 (0.0009) [2023-10-12 21:57:15,815][44958] Updated weights for policy 0, policy_version 46590 (0.0010) [2023-10-12 21:57:16,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 95649792. Throughput: 0: 1637.3, 1: 1672.8. Samples: 23921238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:16,444][43579] Avg episode reward: [(0, '276.870'), (1, '278.820')] [2023-10-12 21:57:17,760][44959] Updated weights for policy 1, policy_version 46820 (0.0009) [2023-10-12 21:57:18,161][44959] Updated weights for policy 1, policy_version 46830 (0.0007) [2023-10-12 21:57:18,523][44959] Updated weights for policy 1, policy_version 46840 (0.0008) [2023-10-12 21:57:19,891][44958] Updated weights for policy 0, policy_version 46600 (0.0010) [2023-10-12 21:57:20,268][44958] Updated weights for policy 0, policy_version 46610 (0.0009) [2023-10-12 21:57:20,636][44958] Updated weights for policy 0, policy_version 46620 (0.0009) [2023-10-12 21:57:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 95715328. Throughput: 0: 1637.5, 1: 1657.4. Samples: 23931316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:21,444][43579] Avg episode reward: [(0, '274.440'), (1, '278.860')] [2023-10-12 21:57:22,706][44959] Updated weights for policy 1, policy_version 46850 (0.0008) [2023-10-12 21:57:23,073][44959] Updated weights for policy 1, policy_version 46860 (0.0009) [2023-10-12 21:57:23,442][44959] Updated weights for policy 1, policy_version 46870 (0.0009) [2023-10-12 21:57:23,812][44959] Updated weights for policy 1, policy_version 46880 (0.0007) [2023-10-12 21:57:24,994][44958] Updated weights for policy 0, policy_version 46630 (0.0011) [2023-10-12 21:57:25,372][44958] Updated weights for policy 0, policy_version 46640 (0.0011) [2023-10-12 21:57:25,746][44958] Updated weights for policy 0, policy_version 46650 (0.0012) [2023-10-12 21:57:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 95780864. Throughput: 0: 1637.2, 1: 1660.9. Samples: 23950946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:26,444][43579] Avg episode reward: [(0, '279.800'), (1, '281.620')] [2023-10-12 21:57:28,076][44959] Updated weights for policy 1, policy_version 46890 (0.0008) [2023-10-12 21:57:28,451][44959] Updated weights for policy 1, policy_version 46900 (0.0008) [2023-10-12 21:57:28,807][44959] Updated weights for policy 1, policy_version 46910 (0.0009) [2023-10-12 21:57:29,985][44958] Updated weights for policy 0, policy_version 46660 (0.0009) [2023-10-12 21:57:30,355][44958] Updated weights for policy 0, policy_version 46670 (0.0008) [2023-10-12 21:57:30,733][44958] Updated weights for policy 0, policy_version 46680 (0.0010) [2023-10-12 21:57:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 95846400. Throughput: 0: 1630.4, 1: 1657.5. Samples: 23970344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:31,444][43579] Avg episode reward: [(0, '278.390'), (1, '283.240')] [2023-10-12 21:57:32,764][44959] Updated weights for policy 1, policy_version 46920 (0.0010) [2023-10-12 21:57:33,136][44959] Updated weights for policy 1, policy_version 46930 (0.0008) [2023-10-12 21:57:33,501][44959] Updated weights for policy 1, policy_version 46940 (0.0007) [2023-10-12 21:57:35,073][44958] Updated weights for policy 0, policy_version 46690 (0.0009) [2023-10-12 21:57:35,443][44958] Updated weights for policy 0, policy_version 46700 (0.0010) [2023-10-12 21:57:35,815][44958] Updated weights for policy 0, policy_version 46710 (0.0009) [2023-10-12 21:57:36,176][44958] Updated weights for policy 0, policy_version 46720 (0.0010) [2023-10-12 21:57:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 95911936. Throughput: 0: 1626.6, 1: 1655.5. Samples: 23980302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:36,443][43579] Avg episode reward: [(0, '282.650'), (1, '286.840')] [2023-10-12 21:57:37,701][44959] Updated weights for policy 1, policy_version 46950 (0.0008) [2023-10-12 21:57:38,061][44959] Updated weights for policy 1, policy_version 46960 (0.0009) [2023-10-12 21:57:38,437][44959] Updated weights for policy 1, policy_version 46970 (0.0009) [2023-10-12 21:57:40,347][44958] Updated weights for policy 0, policy_version 46730 (0.0009) [2023-10-12 21:57:40,725][44958] Updated weights for policy 0, policy_version 46740 (0.0007) [2023-10-12 21:57:41,102][44958] Updated weights for policy 0, policy_version 46750 (0.0009) [2023-10-12 21:57:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 95977472. Throughput: 0: 1638.3, 1: 1654.8. Samples: 24000328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:41,444][43579] Avg episode reward: [(0, '282.280'), (1, '287.270')] [2023-10-12 21:57:42,487][44959] Updated weights for policy 1, policy_version 46980 (0.0008) [2023-10-12 21:57:42,852][44959] Updated weights for policy 1, policy_version 46990 (0.0007) [2023-10-12 21:57:43,226][44959] Updated weights for policy 1, policy_version 47000 (0.0009) [2023-10-12 21:57:45,145][44958] Updated weights for policy 0, policy_version 46760 (0.0008) [2023-10-12 21:57:45,516][44958] Updated weights for policy 0, policy_version 46770 (0.0007) [2023-10-12 21:57:45,883][44958] Updated weights for policy 0, policy_version 46780 (0.0009) [2023-10-12 21:57:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96043008. Throughput: 0: 1632.9, 1: 1657.4. Samples: 24019704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:46,443][43579] Avg episode reward: [(0, '283.150'), (1, '282.070')] [2023-10-12 21:57:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000046784_47906816.pth... [2023-10-12 21:57:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000047008_48136192.pth... [2023-10-12 21:57:46,482][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000045248_46333952.pth [2023-10-12 21:57:46,491][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000045472_46563328.pth [2023-10-12 21:57:47,355][44959] Updated weights for policy 1, policy_version 47010 (0.0008) [2023-10-12 21:57:47,727][44959] Updated weights for policy 1, policy_version 47020 (0.0007) [2023-10-12 21:57:48,092][44959] Updated weights for policy 1, policy_version 47030 (0.0007) [2023-10-12 21:57:48,463][44959] Updated weights for policy 1, policy_version 47040 (0.0008) [2023-10-12 21:57:50,154][44958] Updated weights for policy 0, policy_version 46790 (0.0010) [2023-10-12 21:57:50,533][44958] Updated weights for policy 0, policy_version 46800 (0.0010) [2023-10-12 21:57:50,910][44958] Updated weights for policy 0, policy_version 46810 (0.0007) [2023-10-12 21:57:51,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96108544. Throughput: 0: 1629.4, 1: 1659.0. Samples: 24029822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:51,443][43579] Avg episode reward: [(0, '285.750'), (1, '283.120')] [2023-10-12 21:57:52,701][44959] Updated weights for policy 1, policy_version 47050 (0.0008) [2023-10-12 21:57:53,067][44959] Updated weights for policy 1, policy_version 47060 (0.0009) [2023-10-12 21:57:53,427][44959] Updated weights for policy 1, policy_version 47070 (0.0008) [2023-10-12 21:57:55,097][44958] Updated weights for policy 0, policy_version 46820 (0.0008) [2023-10-12 21:57:55,468][44958] Updated weights for policy 0, policy_version 46830 (0.0010) [2023-10-12 21:57:55,841][44958] Updated weights for policy 0, policy_version 46840 (0.0008) [2023-10-12 21:57:56,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 96174080. Throughput: 0: 1639.3, 1: 1651.6. Samples: 24049556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:57:56,444][43579] Avg episode reward: [(0, '286.180'), (1, '282.450')] [2023-10-12 21:57:56,445][44518] Saving new best policy, reward=286.180! [2023-10-12 21:57:57,649][44959] Updated weights for policy 1, policy_version 47080 (0.0008) [2023-10-12 21:57:58,001][44959] Updated weights for policy 1, policy_version 47090 (0.0008) [2023-10-12 21:57:58,372][44959] Updated weights for policy 1, policy_version 47100 (0.0007) [2023-10-12 21:58:00,232][44958] Updated weights for policy 0, policy_version 46850 (0.0009) [2023-10-12 21:58:00,618][44958] Updated weights for policy 0, policy_version 46860 (0.0007) [2023-10-12 21:58:00,993][44958] Updated weights for policy 0, policy_version 46870 (0.0007) [2023-10-12 21:58:01,362][44958] Updated weights for policy 0, policy_version 46880 (0.0007) [2023-10-12 21:58:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 96239616. Throughput: 0: 1634.1, 1: 1644.3. Samples: 24068766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:58:01,443][43579] Avg episode reward: [(0, '284.550'), (1, '283.830')] [2023-10-12 21:58:02,546][44959] Updated weights for policy 1, policy_version 47110 (0.0009) [2023-10-12 21:58:02,921][44959] Updated weights for policy 1, policy_version 47120 (0.0009) [2023-10-12 21:58:03,283][44959] Updated weights for policy 1, policy_version 47130 (0.0009) [2023-10-12 21:58:05,155][44958] Updated weights for policy 0, policy_version 46890 (0.0008) [2023-10-12 21:58:05,528][44958] Updated weights for policy 0, policy_version 46900 (0.0007) [2023-10-12 21:58:05,910][44958] Updated weights for policy 0, policy_version 46910 (0.0008) [2023-10-12 21:58:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96305152. Throughput: 0: 1627.3, 1: 1648.0. Samples: 24078706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:58:06,444][43579] Avg episode reward: [(0, '276.490'), (1, '283.700')] [2023-10-12 21:58:07,735][44959] Updated weights for policy 1, policy_version 47140 (0.0012) [2023-10-12 21:58:08,119][44959] Updated weights for policy 1, policy_version 47150 (0.0009) [2023-10-12 21:58:08,485][44959] Updated weights for policy 1, policy_version 47160 (0.0011) [2023-10-12 21:58:10,259][44958] Updated weights for policy 0, policy_version 46920 (0.0010) [2023-10-12 21:58:10,630][44958] Updated weights for policy 0, policy_version 46930 (0.0010) [2023-10-12 21:58:11,006][44958] Updated weights for policy 0, policy_version 46940 (0.0007) [2023-10-12 21:58:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96370688. Throughput: 0: 1636.4, 1: 1642.3. Samples: 24098488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:58:11,444][43579] Avg episode reward: [(0, '272.630'), (1, '285.070')] [2023-10-12 21:58:12,399][44959] Updated weights for policy 1, policy_version 47170 (0.0008) [2023-10-12 21:58:12,758][44959] Updated weights for policy 1, policy_version 47180 (0.0007) [2023-10-12 21:58:13,123][44959] Updated weights for policy 1, policy_version 47190 (0.0007) [2023-10-12 21:58:13,480][44959] Updated weights for policy 1, policy_version 47200 (0.0007) [2023-10-12 21:58:15,085][44958] Updated weights for policy 0, policy_version 46950 (0.0008) [2023-10-12 21:58:15,464][44958] Updated weights for policy 0, policy_version 46960 (0.0008) [2023-10-12 21:58:15,846][44958] Updated weights for policy 0, policy_version 46970 (0.0009) [2023-10-12 21:58:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 96436224. Throughput: 0: 1634.0, 1: 1646.6. Samples: 24117972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:58:16,443][43579] Avg episode reward: [(0, '271.130'), (1, '288.170')] [2023-10-12 21:58:17,447][44959] Updated weights for policy 1, policy_version 47210 (0.0010) [2023-10-12 21:58:17,809][44959] Updated weights for policy 1, policy_version 47220 (0.0010) [2023-10-12 21:58:18,178][44959] Updated weights for policy 1, policy_version 47230 (0.0007) [2023-10-12 21:58:20,048][44958] Updated weights for policy 0, policy_version 46980 (0.0009) [2023-10-12 21:58:20,426][44958] Updated weights for policy 0, policy_version 46990 (0.0008) [2023-10-12 21:58:20,792][44958] Updated weights for policy 0, policy_version 47000 (0.0009) [2023-10-12 21:58:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96501760. Throughput: 0: 1637.6, 1: 1648.5. Samples: 24128176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 21:58:21,444][43579] Avg episode reward: [(0, '270.890'), (1, '288.800')] [2023-10-12 21:58:22,425][44959] Updated weights for policy 1, policy_version 47240 (0.0007) [2023-10-12 21:58:22,789][44959] Updated weights for policy 1, policy_version 47250 (0.0009) [2023-10-12 21:58:23,165][44959] Updated weights for policy 1, policy_version 47260 (0.0009) [2023-10-12 21:58:24,853][44958] Updated weights for policy 0, policy_version 47010 (0.0009) [2023-10-12 21:58:25,217][44958] Updated weights for policy 0, policy_version 47020 (0.0010) [2023-10-12 21:58:25,588][44958] Updated weights for policy 0, policy_version 47030 (0.0011) [2023-10-12 21:58:25,967][44958] Updated weights for policy 0, policy_version 47040 (0.0010) [2023-10-12 21:58:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96567296. Throughput: 0: 1637.7, 1: 1645.1. Samples: 24148052. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 21:58:26,443][43579] Avg episode reward: [(0, '269.680'), (1, '291.460')] [2023-10-12 21:58:26,445][44583] Saving new best policy, reward=291.460! [2023-10-12 21:58:27,298][44959] Updated weights for policy 1, policy_version 47270 (0.0008) [2023-10-12 21:58:27,665][44959] Updated weights for policy 1, policy_version 47280 (0.0010) [2023-10-12 21:58:28,036][44959] Updated weights for policy 1, policy_version 47290 (0.0007) [2023-10-12 21:58:30,227][44958] Updated weights for policy 0, policy_version 47050 (0.0010) [2023-10-12 21:58:30,599][44958] Updated weights for policy 0, policy_version 47060 (0.0009) [2023-10-12 21:58:30,979][44958] Updated weights for policy 0, policy_version 47070 (0.0008) [2023-10-12 21:58:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96632832. Throughput: 0: 1638.2, 1: 1647.5. Samples: 24167564. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 21:58:31,444][43579] Avg episode reward: [(0, '274.960'), (1, '288.370')] [2023-10-12 21:58:32,281][44959] Updated weights for policy 1, policy_version 47300 (0.0009) [2023-10-12 21:58:32,646][44959] Updated weights for policy 1, policy_version 47310 (0.0007) [2023-10-12 21:58:33,016][44959] Updated weights for policy 1, policy_version 47320 (0.0007) [2023-10-12 21:58:35,344][44958] Updated weights for policy 0, policy_version 47080 (0.0008) [2023-10-12 21:58:35,708][44958] Updated weights for policy 0, policy_version 47090 (0.0008) [2023-10-12 21:58:36,085][44958] Updated weights for policy 0, policy_version 47100 (0.0007) [2023-10-12 21:58:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96698368. Throughput: 0: 1637.6, 1: 1644.5. Samples: 24177518. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 21:58:36,444][43579] Avg episode reward: [(0, '275.980'), (1, '287.420')] [2023-10-12 21:58:37,302][44959] Updated weights for policy 1, policy_version 47330 (0.0007) [2023-10-12 21:58:37,673][44959] Updated weights for policy 1, policy_version 47340 (0.0009) [2023-10-12 21:58:38,043][44959] Updated weights for policy 1, policy_version 47350 (0.0007) [2023-10-12 21:58:38,406][44959] Updated weights for policy 1, policy_version 47360 (0.0009) [2023-10-12 21:58:39,879][44958] Updated weights for policy 0, policy_version 47110 (0.0007) [2023-10-12 21:58:40,247][44958] Updated weights for policy 0, policy_version 47120 (0.0008) [2023-10-12 21:58:40,619][44958] Updated weights for policy 0, policy_version 47130 (0.0007) [2023-10-12 21:58:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96763904. Throughput: 0: 1637.4, 1: 1646.4. Samples: 24197326. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 21:58:41,444][43579] Avg episode reward: [(0, '281.030'), (1, '280.210')] [2023-10-12 21:58:42,497][44959] Updated weights for policy 1, policy_version 47370 (0.0007) [2023-10-12 21:58:42,873][44959] Updated weights for policy 1, policy_version 47380 (0.0007) [2023-10-12 21:58:43,246][44959] Updated weights for policy 1, policy_version 47390 (0.0008) [2023-10-12 21:58:44,824][44958] Updated weights for policy 0, policy_version 47140 (0.0009) [2023-10-12 21:58:45,209][44958] Updated weights for policy 0, policy_version 47150 (0.0009) [2023-10-12 21:58:45,575][44958] Updated weights for policy 0, policy_version 47160 (0.0008) [2023-10-12 21:58:46,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 96829440. Throughput: 0: 1638.7, 1: 1652.7. Samples: 24216876. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 21:58:46,443][43579] Avg episode reward: [(0, '280.020'), (1, '280.590')] [2023-10-12 21:58:47,419][44959] Updated weights for policy 1, policy_version 47400 (0.0009) [2023-10-12 21:58:47,795][44959] Updated weights for policy 1, policy_version 47410 (0.0007) [2023-10-12 21:58:48,158][44959] Updated weights for policy 1, policy_version 47420 (0.0009) [2023-10-12 21:58:49,835][44958] Updated weights for policy 0, policy_version 47170 (0.0007) [2023-10-12 21:58:50,211][44958] Updated weights for policy 0, policy_version 47180 (0.0007) [2023-10-12 21:58:50,581][44958] Updated weights for policy 0, policy_version 47190 (0.0009) [2023-10-12 21:58:50,955][44958] Updated weights for policy 0, policy_version 47200 (0.0009) [2023-10-12 21:58:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 96894976. Throughput: 0: 1638.2, 1: 1650.5. Samples: 24226700. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 21:58:51,444][43579] Avg episode reward: [(0, '278.440'), (1, '275.450')] [2023-10-12 21:58:52,298][44959] Updated weights for policy 1, policy_version 47430 (0.0009) [2023-10-12 21:58:52,676][44959] Updated weights for policy 1, policy_version 47440 (0.0008) [2023-10-12 21:58:53,045][44959] Updated weights for policy 1, policy_version 47450 (0.0011) [2023-10-12 21:58:55,038][44958] Updated weights for policy 0, policy_version 47210 (0.0008) [2023-10-12 21:58:55,402][44958] Updated weights for policy 0, policy_version 47220 (0.0009) [2023-10-12 21:58:55,784][44958] Updated weights for policy 0, policy_version 47230 (0.0008) [2023-10-12 21:58:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 96960512. Throughput: 0: 1636.9, 1: 1660.4. Samples: 24246868. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 21:58:56,443][43579] Avg episode reward: [(0, '280.400'), (1, '271.920')] [2023-10-12 21:58:57,028][44959] Updated weights for policy 1, policy_version 47460 (0.0010) [2023-10-12 21:58:57,395][44959] Updated weights for policy 1, policy_version 47470 (0.0009) [2023-10-12 21:58:57,763][44959] Updated weights for policy 1, policy_version 47480 (0.0008) [2023-10-12 21:59:00,103][44958] Updated weights for policy 0, policy_version 47240 (0.0008) [2023-10-12 21:59:00,480][44958] Updated weights for policy 0, policy_version 47250 (0.0007) [2023-10-12 21:59:00,846][44958] Updated weights for policy 0, policy_version 47260 (0.0009) [2023-10-12 21:59:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 97026048. Throughput: 0: 1642.0, 1: 1654.9. Samples: 24266334. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-12 21:59:01,444][43579] Avg episode reward: [(0, '280.190'), (1, '275.410')] [2023-10-12 21:59:01,978][44959] Updated weights for policy 1, policy_version 47490 (0.0008) [2023-10-12 21:59:02,352][44959] Updated weights for policy 1, policy_version 47500 (0.0007) [2023-10-12 21:59:02,722][44959] Updated weights for policy 1, policy_version 47510 (0.0007) [2023-10-12 21:59:03,096][44959] Updated weights for policy 1, policy_version 47520 (0.0007) [2023-10-12 21:59:05,049][44958] Updated weights for policy 0, policy_version 47270 (0.0008) [2023-10-12 21:59:05,415][44958] Updated weights for policy 0, policy_version 47280 (0.0009) [2023-10-12 21:59:05,792][44958] Updated weights for policy 0, policy_version 47290 (0.0008) [2023-10-12 21:59:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97091584. Throughput: 0: 1639.5, 1: 1654.2. Samples: 24276392. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-12 21:59:06,443][43579] Avg episode reward: [(0, '279.600'), (1, '273.720')] [2023-10-12 21:59:07,215][44959] Updated weights for policy 1, policy_version 47530 (0.0010) [2023-10-12 21:59:07,572][44959] Updated weights for policy 1, policy_version 47540 (0.0008) [2023-10-12 21:59:07,944][44959] Updated weights for policy 1, policy_version 47550 (0.0007) [2023-10-12 21:59:10,102][44958] Updated weights for policy 0, policy_version 47300 (0.0010) [2023-10-12 21:59:10,477][44958] Updated weights for policy 0, policy_version 47310 (0.0009) [2023-10-12 21:59:10,840][44958] Updated weights for policy 0, policy_version 47320 (0.0008) [2023-10-12 21:59:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97157120. Throughput: 0: 1642.2, 1: 1661.7. Samples: 24296728. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-12 21:59:11,444][43579] Avg episode reward: [(0, '277.490'), (1, '279.570')] [2023-10-12 21:59:12,071][44959] Updated weights for policy 1, policy_version 47560 (0.0010) [2023-10-12 21:59:12,440][44959] Updated weights for policy 1, policy_version 47570 (0.0009) [2023-10-12 21:59:12,811][44959] Updated weights for policy 1, policy_version 47580 (0.0010) [2023-10-12 21:59:15,010][44958] Updated weights for policy 0, policy_version 47330 (0.0008) [2023-10-12 21:59:15,389][44958] Updated weights for policy 0, policy_version 47340 (0.0008) [2023-10-12 21:59:15,757][44958] Updated weights for policy 0, policy_version 47350 (0.0009) [2023-10-12 21:59:16,121][44958] Updated weights for policy 0, policy_version 47360 (0.0007) [2023-10-12 21:59:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97222656. Throughput: 0: 1643.1, 1: 1655.5. Samples: 24316002. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-12 21:59:16,443][43579] Avg episode reward: [(0, '280.410'), (1, '276.160')] [2023-10-12 21:59:16,890][44959] Updated weights for policy 1, policy_version 47590 (0.0008) [2023-10-12 21:59:17,266][44959] Updated weights for policy 1, policy_version 47600 (0.0009) [2023-10-12 21:59:17,646][44959] Updated weights for policy 1, policy_version 47610 (0.0008) [2023-10-12 21:59:20,270][44958] Updated weights for policy 0, policy_version 47370 (0.0008) [2023-10-12 21:59:20,636][44958] Updated weights for policy 0, policy_version 47380 (0.0008) [2023-10-12 21:59:21,017][44958] Updated weights for policy 0, policy_version 47390 (0.0009) [2023-10-12 21:59:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97288192. Throughput: 0: 1644.2, 1: 1655.3. Samples: 24325996. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-12 21:59:21,444][43579] Avg episode reward: [(0, '283.770'), (1, '278.360')] [2023-10-12 21:59:21,854][44959] Updated weights for policy 1, policy_version 47620 (0.0009) [2023-10-12 21:59:22,216][44959] Updated weights for policy 1, policy_version 47630 (0.0008) [2023-10-12 21:59:22,574][44959] Updated weights for policy 1, policy_version 47640 (0.0009) [2023-10-12 21:59:25,168][44958] Updated weights for policy 0, policy_version 47400 (0.0009) [2023-10-12 21:59:25,538][44958] Updated weights for policy 0, policy_version 47410 (0.0008) [2023-10-12 21:59:25,902][44958] Updated weights for policy 0, policy_version 47420 (0.0007) [2023-10-12 21:59:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97353728. Throughput: 0: 1645.6, 1: 1662.6. Samples: 24346194. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-12 21:59:26,444][43579] Avg episode reward: [(0, '282.570'), (1, '277.330')] [2023-10-12 21:59:26,752][44959] Updated weights for policy 1, policy_version 47650 (0.0007) [2023-10-12 21:59:27,119][44959] Updated weights for policy 1, policy_version 47660 (0.0008) [2023-10-12 21:59:27,478][44959] Updated weights for policy 1, policy_version 47670 (0.0008) [2023-10-12 21:59:27,846][44959] Updated weights for policy 1, policy_version 47680 (0.0011) [2023-10-12 21:59:30,280][44958] Updated weights for policy 0, policy_version 47430 (0.0008) [2023-10-12 21:59:30,663][44958] Updated weights for policy 0, policy_version 47440 (0.0007) [2023-10-12 21:59:31,027][44958] Updated weights for policy 0, policy_version 47450 (0.0008) [2023-10-12 21:59:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97419264. Throughput: 0: 1643.1, 1: 1658.8. Samples: 24365460. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-12 21:59:31,444][43579] Avg episode reward: [(0, '280.080'), (1, '276.870')] [2023-10-12 21:59:31,929][44959] Updated weights for policy 1, policy_version 47690 (0.0009) [2023-10-12 21:59:32,302][44959] Updated weights for policy 1, policy_version 47700 (0.0009) [2023-10-12 21:59:32,656][44959] Updated weights for policy 1, policy_version 47710 (0.0011) [2023-10-12 21:59:35,135][44958] Updated weights for policy 0, policy_version 47460 (0.0008) [2023-10-12 21:59:35,516][44958] Updated weights for policy 0, policy_version 47470 (0.0008) [2023-10-12 21:59:35,880][44958] Updated weights for policy 0, policy_version 47480 (0.0008) [2023-10-12 21:59:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 97484800. Throughput: 0: 1642.4, 1: 1663.2. Samples: 24375450. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 21:59:36,443][43579] Avg episode reward: [(0, '277.000'), (1, '273.690')] [2023-10-12 21:59:36,800][44959] Updated weights for policy 1, policy_version 47720 (0.0009) [2023-10-12 21:59:37,171][44959] Updated weights for policy 1, policy_version 47730 (0.0010) [2023-10-12 21:59:37,525][44959] Updated weights for policy 1, policy_version 47740 (0.0009) [2023-10-12 21:59:39,918][44958] Updated weights for policy 0, policy_version 47490 (0.0010) [2023-10-12 21:59:40,283][44958] Updated weights for policy 0, policy_version 47500 (0.0010) [2023-10-12 21:59:40,665][44958] Updated weights for policy 0, policy_version 47510 (0.0009) [2023-10-12 21:59:41,041][44958] Updated weights for policy 0, policy_version 47520 (0.0008) [2023-10-12 21:59:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97550336. Throughput: 0: 1648.1, 1: 1661.9. Samples: 24395822. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 21:59:41,444][43579] Avg episode reward: [(0, '275.210'), (1, '275.140')] [2023-10-12 21:59:41,822][44959] Updated weights for policy 1, policy_version 47750 (0.0009) [2023-10-12 21:59:42,209][44959] Updated weights for policy 1, policy_version 47760 (0.0007) [2023-10-12 21:59:42,587][44959] Updated weights for policy 1, policy_version 47770 (0.0008) [2023-10-12 21:59:45,158][44958] Updated weights for policy 0, policy_version 47530 (0.0011) [2023-10-12 21:59:45,521][44958] Updated weights for policy 0, policy_version 47540 (0.0011) [2023-10-12 21:59:45,891][44958] Updated weights for policy 0, policy_version 47550 (0.0009) [2023-10-12 21:59:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97615872. Throughput: 0: 1644.9, 1: 1657.3. Samples: 24414932. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 21:59:46,443][43579] Avg episode reward: [(0, '274.300'), (1, '273.820')] [2023-10-12 21:59:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000047776_48922624.pth... [2023-10-12 21:59:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000047552_48693248.pth... [2023-10-12 21:59:46,487][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000046016_47120384.pth [2023-10-12 21:59:46,494][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000046240_47349760.pth [2023-10-12 21:59:46,782][44959] Updated weights for policy 1, policy_version 47780 (0.0008) [2023-10-12 21:59:47,143][44959] Updated weights for policy 1, policy_version 47790 (0.0010) [2023-10-12 21:59:47,518][44959] Updated weights for policy 1, policy_version 47800 (0.0009) [2023-10-12 21:59:50,052][44958] Updated weights for policy 0, policy_version 47560 (0.0010) [2023-10-12 21:59:50,416][44958] Updated weights for policy 0, policy_version 47570 (0.0007) [2023-10-12 21:59:50,785][44958] Updated weights for policy 0, policy_version 47580 (0.0011) [2023-10-12 21:59:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97681408. Throughput: 0: 1648.1, 1: 1655.4. Samples: 24425052. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 21:59:51,443][43579] Avg episode reward: [(0, '273.680'), (1, '280.100')] [2023-10-12 21:59:51,592][44959] Updated weights for policy 1, policy_version 47810 (0.0009) [2023-10-12 21:59:51,957][44959] Updated weights for policy 1, policy_version 47820 (0.0009) [2023-10-12 21:59:52,315][44959] Updated weights for policy 1, policy_version 47830 (0.0007) [2023-10-12 21:59:52,690][44959] Updated weights for policy 1, policy_version 47840 (0.0007) [2023-10-12 21:59:54,907][44958] Updated weights for policy 0, policy_version 47590 (0.0010) [2023-10-12 21:59:55,269][44958] Updated weights for policy 0, policy_version 47600 (0.0007) [2023-10-12 21:59:55,642][44958] Updated weights for policy 0, policy_version 47610 (0.0007) [2023-10-12 21:59:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97746944. Throughput: 0: 1640.0, 1: 1647.2. Samples: 24444648. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 21:59:56,443][43579] Avg episode reward: [(0, '272.470'), (1, '280.160')] [2023-10-12 21:59:56,844][44959] Updated weights for policy 1, policy_version 47850 (0.0011) [2023-10-12 21:59:57,221][44959] Updated weights for policy 1, policy_version 47860 (0.0008) [2023-10-12 21:59:57,600][44959] Updated weights for policy 1, policy_version 47870 (0.0008) [2023-10-12 21:59:59,963][44958] Updated weights for policy 0, policy_version 47620 (0.0010) [2023-10-12 22:00:00,339][44958] Updated weights for policy 0, policy_version 47630 (0.0009) [2023-10-12 22:00:00,709][44958] Updated weights for policy 0, policy_version 47640 (0.0007) [2023-10-12 22:00:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97812480. Throughput: 0: 1639.5, 1: 1649.1. Samples: 24463990. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 22:00:01,444][43579] Avg episode reward: [(0, '275.540'), (1, '282.680')] [2023-10-12 22:00:01,646][44959] Updated weights for policy 1, policy_version 47880 (0.0008) [2023-10-12 22:00:02,015][44959] Updated weights for policy 1, policy_version 47890 (0.0009) [2023-10-12 22:00:02,381][44959] Updated weights for policy 1, policy_version 47900 (0.0008) [2023-10-12 22:00:04,931][44958] Updated weights for policy 0, policy_version 47650 (0.0009) [2023-10-12 22:00:05,297][44958] Updated weights for policy 0, policy_version 47660 (0.0007) [2023-10-12 22:00:05,664][44958] Updated weights for policy 0, policy_version 47670 (0.0010) [2023-10-12 22:00:06,038][44958] Updated weights for policy 0, policy_version 47680 (0.0009) [2023-10-12 22:00:06,408][44959] Updated weights for policy 1, policy_version 47910 (0.0008) [2023-10-12 22:00:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97878016. Throughput: 0: 1639.1, 1: 1654.4. Samples: 24474202. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 22:00:06,443][43579] Avg episode reward: [(0, '278.260'), (1, '282.630')] [2023-10-12 22:00:06,772][44959] Updated weights for policy 1, policy_version 47920 (0.0008) [2023-10-12 22:00:07,142][44959] Updated weights for policy 1, policy_version 47930 (0.0009) [2023-10-12 22:00:10,233][44958] Updated weights for policy 0, policy_version 47690 (0.0010) [2023-10-12 22:00:10,604][44958] Updated weights for policy 0, policy_version 47700 (0.0011) [2023-10-12 22:00:10,986][44958] Updated weights for policy 0, policy_version 47710 (0.0012) [2023-10-12 22:00:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 97943552. Throughput: 0: 1634.7, 1: 1652.4. Samples: 24494114. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-12 22:00:11,443][43579] Avg episode reward: [(0, '275.750'), (1, '283.950')] [2023-10-12 22:00:11,540][44959] Updated weights for policy 1, policy_version 47940 (0.0007) [2023-10-12 22:00:11,906][44959] Updated weights for policy 1, policy_version 47950 (0.0008) [2023-10-12 22:00:12,275][44959] Updated weights for policy 1, policy_version 47960 (0.0011) [2023-10-12 22:00:15,189][44958] Updated weights for policy 0, policy_version 47720 (0.0010) [2023-10-12 22:00:15,575][44958] Updated weights for policy 0, policy_version 47730 (0.0010) [2023-10-12 22:00:15,954][44958] Updated weights for policy 0, policy_version 47740 (0.0010) [2023-10-12 22:00:16,226][44959] Updated weights for policy 1, policy_version 47970 (0.0010) [2023-10-12 22:00:16,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98009088. Throughput: 0: 1633.3, 1: 1659.4. Samples: 24513630. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-12 22:00:16,443][43579] Avg episode reward: [(0, '275.000'), (1, '284.110')] [2023-10-12 22:00:16,597][44959] Updated weights for policy 1, policy_version 47980 (0.0011) [2023-10-12 22:00:16,962][44959] Updated weights for policy 1, policy_version 47990 (0.0011) [2023-10-12 22:00:17,330][44959] Updated weights for policy 1, policy_version 48000 (0.0009) [2023-10-12 22:00:19,863][44958] Updated weights for policy 0, policy_version 47750 (0.0011) [2023-10-12 22:00:20,233][44958] Updated weights for policy 0, policy_version 47760 (0.0010) [2023-10-12 22:00:20,610][44958] Updated weights for policy 0, policy_version 47770 (0.0010) [2023-10-12 22:00:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98074624. Throughput: 0: 1645.6, 1: 1654.7. Samples: 24523962. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-12 22:00:21,443][43579] Avg episode reward: [(0, '277.340'), (1, '285.480')] [2023-10-12 22:00:21,656][44959] Updated weights for policy 1, policy_version 48010 (0.0009) [2023-10-12 22:00:22,036][44959] Updated weights for policy 1, policy_version 48020 (0.0010) [2023-10-12 22:00:22,411][44959] Updated weights for policy 1, policy_version 48030 (0.0009) [2023-10-12 22:00:24,858][44958] Updated weights for policy 0, policy_version 47780 (0.0009) [2023-10-12 22:00:25,231][44958] Updated weights for policy 0, policy_version 47790 (0.0007) [2023-10-12 22:00:25,601][44958] Updated weights for policy 0, policy_version 47800 (0.0007) [2023-10-12 22:00:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98140160. Throughput: 0: 1633.6, 1: 1642.1. Samples: 24543230. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-12 22:00:26,443][43579] Avg episode reward: [(0, '278.840'), (1, '281.860')] [2023-10-12 22:00:26,686][44959] Updated weights for policy 1, policy_version 48040 (0.0008) [2023-10-12 22:00:27,067][44959] Updated weights for policy 1, policy_version 48050 (0.0008) [2023-10-12 22:00:27,431][44959] Updated weights for policy 1, policy_version 48060 (0.0007) [2023-10-12 22:00:29,910][44958] Updated weights for policy 0, policy_version 47810 (0.0008) [2023-10-12 22:00:30,287][44958] Updated weights for policy 0, policy_version 47820 (0.0008) [2023-10-12 22:00:30,657][44958] Updated weights for policy 0, policy_version 47830 (0.0008) [2023-10-12 22:00:31,023][44958] Updated weights for policy 0, policy_version 47840 (0.0008) [2023-10-12 22:00:31,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98205696. Throughput: 0: 1635.4, 1: 1646.7. Samples: 24562626. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-12 22:00:31,444][43579] Avg episode reward: [(0, '279.980'), (1, '283.120')] [2023-10-12 22:00:31,567][44959] Updated weights for policy 1, policy_version 48070 (0.0007) [2023-10-12 22:00:31,936][44959] Updated weights for policy 1, policy_version 48080 (0.0008) [2023-10-12 22:00:32,298][44959] Updated weights for policy 1, policy_version 48090 (0.0008) [2023-10-12 22:00:35,094][44958] Updated weights for policy 0, policy_version 47850 (0.0011) [2023-10-12 22:00:35,465][44958] Updated weights for policy 0, policy_version 47860 (0.0008) [2023-10-12 22:00:35,847][44958] Updated weights for policy 0, policy_version 47870 (0.0007) [2023-10-12 22:00:36,317][44959] Updated weights for policy 1, policy_version 48100 (0.0009) [2023-10-12 22:00:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98271232. Throughput: 0: 1637.9, 1: 1649.0. Samples: 24572962. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-12 22:00:36,443][43579] Avg episode reward: [(0, '280.020'), (1, '282.080')] [2023-10-12 22:00:36,696][44959] Updated weights for policy 1, policy_version 48110 (0.0007) [2023-10-12 22:00:37,059][44959] Updated weights for policy 1, policy_version 48120 (0.0010) [2023-10-12 22:00:40,062][44958] Updated weights for policy 0, policy_version 47880 (0.0007) [2023-10-12 22:00:40,430][44958] Updated weights for policy 0, policy_version 47890 (0.0008) [2023-10-12 22:00:40,814][44958] Updated weights for policy 0, policy_version 47900 (0.0009) [2023-10-12 22:00:41,146][44959] Updated weights for policy 1, policy_version 48130 (0.0010) [2023-10-12 22:00:41,443][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98336768. Throughput: 0: 1639.1, 1: 1650.7. Samples: 24592686. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-12 22:00:41,443][43579] Avg episode reward: [(0, '284.070'), (1, '281.310')] [2023-10-12 22:00:41,522][44959] Updated weights for policy 1, policy_version 48140 (0.0010) [2023-10-12 22:00:41,902][44959] Updated weights for policy 1, policy_version 48150 (0.0008) [2023-10-12 22:00:42,271][44959] Updated weights for policy 1, policy_version 48160 (0.0009) [2023-10-12 22:00:45,100][44958] Updated weights for policy 0, policy_version 47910 (0.0008) [2023-10-12 22:00:45,474][44958] Updated weights for policy 0, policy_version 47920 (0.0007) [2023-10-12 22:00:45,842][44958] Updated weights for policy 0, policy_version 47930 (0.0007) [2023-10-12 22:00:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98402304. Throughput: 0: 1638.8, 1: 1657.5. Samples: 24612326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:00:46,444][43579] Avg episode reward: [(0, '283.600'), (1, '278.560')] [2023-10-12 22:00:46,482][44959] Updated weights for policy 1, policy_version 48170 (0.0007) [2023-10-12 22:00:46,855][44959] Updated weights for policy 1, policy_version 48180 (0.0009) [2023-10-12 22:00:47,220][44959] Updated weights for policy 1, policy_version 48190 (0.0007) [2023-10-12 22:00:49,952][44958] Updated weights for policy 0, policy_version 47940 (0.0007) [2023-10-12 22:00:50,333][44958] Updated weights for policy 0, policy_version 47950 (0.0007) [2023-10-12 22:00:50,705][44958] Updated weights for policy 0, policy_version 47960 (0.0009) [2023-10-12 22:00:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98467840. Throughput: 0: 1638.1, 1: 1652.6. Samples: 24622282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:00:51,444][43579] Avg episode reward: [(0, '281.700'), (1, '274.190')] [2023-10-12 22:00:51,511][44959] Updated weights for policy 1, policy_version 48200 (0.0009) [2023-10-12 22:00:51,886][44959] Updated weights for policy 1, policy_version 48210 (0.0007) [2023-10-12 22:00:52,256][44959] Updated weights for policy 1, policy_version 48220 (0.0011) [2023-10-12 22:00:55,040][44958] Updated weights for policy 0, policy_version 47970 (0.0009) [2023-10-12 22:00:55,412][44958] Updated weights for policy 0, policy_version 47980 (0.0007) [2023-10-12 22:00:55,786][44958] Updated weights for policy 0, policy_version 47990 (0.0008) [2023-10-12 22:00:56,144][44959] Updated weights for policy 1, policy_version 48230 (0.0010) [2023-10-12 22:00:56,164][44958] Updated weights for policy 0, policy_version 48000 (0.0008) [2023-10-12 22:00:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98533376. Throughput: 0: 1641.4, 1: 1649.9. Samples: 24642222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:00:56,444][43579] Avg episode reward: [(0, '282.590'), (1, '276.550')] [2023-10-12 22:00:56,512][44959] Updated weights for policy 1, policy_version 48240 (0.0008) [2023-10-12 22:00:56,882][44959] Updated weights for policy 1, policy_version 48250 (0.0007) [2023-10-12 22:01:00,175][44958] Updated weights for policy 0, policy_version 48010 (0.0009) [2023-10-12 22:01:00,549][44958] Updated weights for policy 0, policy_version 48020 (0.0011) [2023-10-12 22:01:00,919][44958] Updated weights for policy 0, policy_version 48030 (0.0011) [2023-10-12 22:01:01,036][44959] Updated weights for policy 1, policy_version 48260 (0.0008) [2023-10-12 22:01:01,409][44959] Updated weights for policy 1, policy_version 48270 (0.0010) [2023-10-12 22:01:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 98598912. Throughput: 0: 1640.1, 1: 1642.9. Samples: 24661366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:01:01,443][43579] Avg episode reward: [(0, '282.900'), (1, '275.530')] [2023-10-12 22:01:01,790][44959] Updated weights for policy 1, policy_version 48280 (0.0009) [2023-10-12 22:01:04,923][44958] Updated weights for policy 0, policy_version 48040 (0.0007) [2023-10-12 22:01:05,294][44958] Updated weights for policy 0, policy_version 48050 (0.0008) [2023-10-12 22:01:05,662][44958] Updated weights for policy 0, policy_version 48060 (0.0010) [2023-10-12 22:01:06,157][44959] Updated weights for policy 1, policy_version 48290 (0.0009) [2023-10-12 22:01:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98664448. Throughput: 0: 1636.9, 1: 1645.9. Samples: 24671686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:01:06,444][43579] Avg episode reward: [(0, '282.150'), (1, '277.180')] [2023-10-12 22:01:06,519][44959] Updated weights for policy 1, policy_version 48300 (0.0009) [2023-10-12 22:01:06,884][44959] Updated weights for policy 1, policy_version 48310 (0.0008) [2023-10-12 22:01:07,251][44959] Updated weights for policy 1, policy_version 48320 (0.0009) [2023-10-12 22:01:09,991][44958] Updated weights for policy 0, policy_version 48070 (0.0008) [2023-10-12 22:01:10,368][44958] Updated weights for policy 0, policy_version 48080 (0.0008) [2023-10-12 22:01:10,737][44958] Updated weights for policy 0, policy_version 48090 (0.0009) [2023-10-12 22:01:11,123][44959] Updated weights for policy 1, policy_version 48330 (0.0007) [2023-10-12 22:01:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98729984. Throughput: 0: 1641.2, 1: 1661.6. Samples: 24691854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:01:11,444][43579] Avg episode reward: [(0, '285.230'), (1, '270.980')] [2023-10-12 22:01:11,491][44959] Updated weights for policy 1, policy_version 48340 (0.0008) [2023-10-12 22:01:11,860][44959] Updated weights for policy 1, policy_version 48350 (0.0008) [2023-10-12 22:01:14,940][44958] Updated weights for policy 0, policy_version 48100 (0.0008) [2023-10-12 22:01:15,313][44958] Updated weights for policy 0, policy_version 48110 (0.0008) [2023-10-12 22:01:15,686][44958] Updated weights for policy 0, policy_version 48120 (0.0009) [2023-10-12 22:01:15,971][44959] Updated weights for policy 1, policy_version 48360 (0.0008) [2023-10-12 22:01:16,327][44959] Updated weights for policy 1, policy_version 48370 (0.0007) [2023-10-12 22:01:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98795520. Throughput: 0: 1640.3, 1: 1652.1. Samples: 24710780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:01:16,444][43579] Avg episode reward: [(0, '281.550'), (1, '270.980')] [2023-10-12 22:01:16,692][44959] Updated weights for policy 1, policy_version 48380 (0.0009) [2023-10-12 22:01:19,847][44958] Updated weights for policy 0, policy_version 48130 (0.0008) [2023-10-12 22:01:20,226][44958] Updated weights for policy 0, policy_version 48140 (0.0009) [2023-10-12 22:01:20,598][44958] Updated weights for policy 0, policy_version 48150 (0.0011) [2023-10-12 22:01:20,969][44958] Updated weights for policy 0, policy_version 48160 (0.0010) [2023-10-12 22:01:21,008][44959] Updated weights for policy 1, policy_version 48390 (0.0010) [2023-10-12 22:01:21,376][44959] Updated weights for policy 1, policy_version 48400 (0.0009) [2023-10-12 22:01:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98861056. Throughput: 0: 1635.7, 1: 1658.3. Samples: 24721194. Policy #0 lag: (min: 10.0, avg: 13.5, max: 42.0) [2023-10-12 22:01:21,443][43579] Avg episode reward: [(0, '281.080'), (1, '273.430')] [2023-10-12 22:01:21,739][44959] Updated weights for policy 1, policy_version 48410 (0.0008) [2023-10-12 22:01:25,211][44958] Updated weights for policy 0, policy_version 48170 (0.0008) [2023-10-12 22:01:25,584][44958] Updated weights for policy 0, policy_version 48180 (0.0008) [2023-10-12 22:01:25,956][44958] Updated weights for policy 0, policy_version 48190 (0.0009) [2023-10-12 22:01:26,031][44959] Updated weights for policy 1, policy_version 48420 (0.0010) [2023-10-12 22:01:26,390][44959] Updated weights for policy 1, policy_version 48430 (0.0010) [2023-10-12 22:01:26,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 98926592. Throughput: 0: 1635.9, 1: 1657.2. Samples: 24740872. Policy #0 lag: (min: 10.0, avg: 13.5, max: 42.0) [2023-10-12 22:01:26,443][43579] Avg episode reward: [(0, '279.080'), (1, '269.930')] [2023-10-12 22:01:26,767][44959] Updated weights for policy 1, policy_version 48440 (0.0008) [2023-10-12 22:01:30,115][44958] Updated weights for policy 0, policy_version 48200 (0.0008) [2023-10-12 22:01:30,492][44958] Updated weights for policy 0, policy_version 48210 (0.0009) [2023-10-12 22:01:30,860][44958] Updated weights for policy 0, policy_version 48220 (0.0008) [2023-10-12 22:01:30,865][44959] Updated weights for policy 1, policy_version 48450 (0.0008) [2023-10-12 22:01:31,221][44959] Updated weights for policy 1, policy_version 48460 (0.0007) [2023-10-12 22:01:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 98992128. Throughput: 0: 1633.8, 1: 1647.3. Samples: 24759976. Policy #0 lag: (min: 10.0, avg: 13.5, max: 42.0) [2023-10-12 22:01:31,443][43579] Avg episode reward: [(0, '278.970'), (1, '271.970')] [2023-10-12 22:01:31,597][44959] Updated weights for policy 1, policy_version 48470 (0.0008) [2023-10-12 22:01:31,964][44959] Updated weights for policy 1, policy_version 48480 (0.0008) [2023-10-12 22:01:35,008][44958] Updated weights for policy 0, policy_version 48230 (0.0007) [2023-10-12 22:01:35,382][44958] Updated weights for policy 0, policy_version 48240 (0.0007) [2023-10-12 22:01:35,757][44958] Updated weights for policy 0, policy_version 48250 (0.0007) [2023-10-12 22:01:35,834][44959] Updated weights for policy 1, policy_version 48490 (0.0007) [2023-10-12 22:01:36,205][44959] Updated weights for policy 1, policy_version 48500 (0.0007) [2023-10-12 22:01:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 99057664. Throughput: 0: 1633.4, 1: 1655.7. Samples: 24770290. Policy #0 lag: (min: 10.0, avg: 13.5, max: 42.0) [2023-10-12 22:01:36,444][43579] Avg episode reward: [(0, '278.480'), (1, '276.630')] [2023-10-12 22:01:36,576][44959] Updated weights for policy 1, policy_version 48510 (0.0008) [2023-10-12 22:01:39,933][44958] Updated weights for policy 0, policy_version 48260 (0.0007) [2023-10-12 22:01:40,308][44958] Updated weights for policy 0, policy_version 48270 (0.0010) [2023-10-12 22:01:40,681][44958] Updated weights for policy 0, policy_version 48280 (0.0007) [2023-10-12 22:01:40,860][44959] Updated weights for policy 1, policy_version 48520 (0.0009) [2023-10-12 22:01:41,227][44959] Updated weights for policy 1, policy_version 48530 (0.0009) [2023-10-12 22:01:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 99123200. Throughput: 0: 1628.6, 1: 1661.0. Samples: 24790254. Policy #0 lag: (min: 10.0, avg: 13.5, max: 42.0) [2023-10-12 22:01:41,443][43579] Avg episode reward: [(0, '276.600'), (1, '278.180')] [2023-10-12 22:01:41,593][44959] Updated weights for policy 1, policy_version 48540 (0.0008) [2023-10-12 22:01:44,944][44958] Updated weights for policy 0, policy_version 48290 (0.0007) [2023-10-12 22:01:45,347][44958] Updated weights for policy 0, policy_version 48300 (0.0008) [2023-10-12 22:01:45,702][44959] Updated weights for policy 1, policy_version 48550 (0.0008) [2023-10-12 22:01:45,713][44958] Updated weights for policy 0, policy_version 48310 (0.0008) [2023-10-12 22:01:46,072][44959] Updated weights for policy 1, policy_version 48560 (0.0008) [2023-10-12 22:01:46,089][44958] Updated weights for policy 0, policy_version 48320 (0.0010) [2023-10-12 22:01:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 99188736. Throughput: 0: 1632.1, 1: 1647.5. Samples: 24808950. Policy #0 lag: (min: 10.0, avg: 13.5, max: 42.0) [2023-10-12 22:01:46,443][43579] Avg episode reward: [(0, '276.050'), (1, '284.190')] [2023-10-12 22:01:46,447][44959] Updated weights for policy 1, policy_version 48570 (0.0010) [2023-10-12 22:01:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000048320_49479680.pth... [2023-10-12 22:01:46,488][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000046784_47906816.pth [2023-10-12 22:01:46,671][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000048576_49741824.pth... [2023-10-12 22:01:46,709][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000047008_48136192.pth [2023-10-12 22:01:50,416][44958] Updated weights for policy 0, policy_version 48330 (0.0007) [2023-10-12 22:01:50,767][44959] Updated weights for policy 1, policy_version 48580 (0.0011) [2023-10-12 22:01:50,785][44958] Updated weights for policy 0, policy_version 48340 (0.0008) [2023-10-12 22:01:51,133][44959] Updated weights for policy 1, policy_version 48590 (0.0010) [2023-10-12 22:01:51,152][44958] Updated weights for policy 0, policy_version 48350 (0.0009) [2023-10-12 22:01:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 99254272. Throughput: 0: 1625.6, 1: 1654.1. Samples: 24819272. Policy #0 lag: (min: 10.0, avg: 13.5, max: 42.0) [2023-10-12 22:01:51,443][43579] Avg episode reward: [(0, '269.580'), (1, '282.770')] [2023-10-12 22:01:51,503][44959] Updated weights for policy 1, policy_version 48600 (0.0007) [2023-10-12 22:01:55,411][44958] Updated weights for policy 0, policy_version 48360 (0.0008) [2023-10-12 22:01:55,782][44958] Updated weights for policy 0, policy_version 48370 (0.0008) [2023-10-12 22:01:55,817][44959] Updated weights for policy 1, policy_version 48610 (0.0009) [2023-10-12 22:01:56,161][44958] Updated weights for policy 0, policy_version 48380 (0.0009) [2023-10-12 22:01:56,208][44959] Updated weights for policy 1, policy_version 48620 (0.0009) [2023-10-12 22:01:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 99319808. Throughput: 0: 1629.6, 1: 1644.9. Samples: 24839208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:01:56,443][43579] Avg episode reward: [(0, '265.480'), (1, '285.210')] [2023-10-12 22:01:56,586][44959] Updated weights for policy 1, policy_version 48630 (0.0010) [2023-10-12 22:01:56,945][44959] Updated weights for policy 1, policy_version 48640 (0.0011) [2023-10-12 22:02:00,310][44958] Updated weights for policy 0, policy_version 48390 (0.0009) [2023-10-12 22:02:00,686][44958] Updated weights for policy 0, policy_version 48400 (0.0007) [2023-10-12 22:02:00,970][44959] Updated weights for policy 1, policy_version 48650 (0.0007) [2023-10-12 22:02:01,057][44958] Updated weights for policy 0, policy_version 48410 (0.0009) [2023-10-12 22:02:01,329][44959] Updated weights for policy 1, policy_version 48660 (0.0009) [2023-10-12 22:02:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 99385344. Throughput: 0: 1628.3, 1: 1643.4. Samples: 24858004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:02:01,443][43579] Avg episode reward: [(0, '261.610'), (1, '285.720')] [2023-10-12 22:02:01,698][44959] Updated weights for policy 1, policy_version 48670 (0.0007) [2023-10-12 22:02:05,173][44958] Updated weights for policy 0, policy_version 48420 (0.0008) [2023-10-12 22:02:05,475][44959] Updated weights for policy 1, policy_version 48680 (0.0007) [2023-10-12 22:02:05,550][44958] Updated weights for policy 0, policy_version 48430 (0.0008) [2023-10-12 22:02:05,855][44959] Updated weights for policy 1, policy_version 48690 (0.0008) [2023-10-12 22:02:05,925][44958] Updated weights for policy 0, policy_version 48440 (0.0007) [2023-10-12 22:02:06,213][44959] Updated weights for policy 1, policy_version 48700 (0.0007) [2023-10-12 22:02:06,442][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 99483648. Throughput: 0: 1620.4, 1: 1649.0. Samples: 24868318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:02:06,443][43579] Avg episode reward: [(0, '262.430'), (1, '287.990')] [2023-10-12 22:02:10,044][44958] Updated weights for policy 0, policy_version 48450 (0.0008) [2023-10-12 22:02:10,354][44959] Updated weights for policy 1, policy_version 48710 (0.0007) [2023-10-12 22:02:10,414][44958] Updated weights for policy 0, policy_version 48460 (0.0007) [2023-10-12 22:02:10,722][44959] Updated weights for policy 1, policy_version 48720 (0.0007) [2023-10-12 22:02:10,783][44958] Updated weights for policy 0, policy_version 48470 (0.0007) [2023-10-12 22:02:11,101][44959] Updated weights for policy 1, policy_version 48730 (0.0009) [2023-10-12 22:02:11,150][44958] Updated weights for policy 0, policy_version 48480 (0.0007) [2023-10-12 22:02:11,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 99549184. Throughput: 0: 1630.6, 1: 1661.5. Samples: 24889016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:02:11,444][43579] Avg episode reward: [(0, '265.780'), (1, '284.260')] [2023-10-12 22:02:15,255][44958] Updated weights for policy 0, policy_version 48490 (0.0008) [2023-10-12 22:02:15,260][44959] Updated weights for policy 1, policy_version 48740 (0.0008) [2023-10-12 22:02:15,628][44959] Updated weights for policy 1, policy_version 48750 (0.0010) [2023-10-12 22:02:15,632][44958] Updated weights for policy 0, policy_version 48500 (0.0009) [2023-10-12 22:02:15,992][44959] Updated weights for policy 1, policy_version 48760 (0.0008) [2023-10-12 22:02:16,001][44958] Updated weights for policy 0, policy_version 48510 (0.0007) [2023-10-12 22:02:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 99614720. Throughput: 0: 1627.3, 1: 1647.2. Samples: 24907332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:02:16,444][43579] Avg episode reward: [(0, '270.920'), (1, '282.430')] [2023-10-12 22:02:20,067][44959] Updated weights for policy 1, policy_version 48770 (0.0009) [2023-10-12 22:02:20,144][44958] Updated weights for policy 0, policy_version 48520 (0.0009) [2023-10-12 22:02:20,426][44959] Updated weights for policy 1, policy_version 48780 (0.0009) [2023-10-12 22:02:20,516][44958] Updated weights for policy 0, policy_version 48530 (0.0010) [2023-10-12 22:02:20,798][44959] Updated weights for policy 1, policy_version 48790 (0.0009) [2023-10-12 22:02:20,889][44958] Updated weights for policy 0, policy_version 48540 (0.0008) [2023-10-12 22:02:21,172][44959] Updated weights for policy 1, policy_version 48800 (0.0007) [2023-10-12 22:02:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 99680256. Throughput: 0: 1630.8, 1: 1662.7. Samples: 24918498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:02:21,443][43579] Avg episode reward: [(0, '276.740'), (1, '276.290')] [2023-10-12 22:02:24,980][44958] Updated weights for policy 0, policy_version 48550 (0.0009) [2023-10-12 22:02:25,356][44958] Updated weights for policy 0, policy_version 48560 (0.0007) [2023-10-12 22:02:25,388][44959] Updated weights for policy 1, policy_version 48810 (0.0007) [2023-10-12 22:02:25,726][44958] Updated weights for policy 0, policy_version 48570 (0.0008) [2023-10-12 22:02:25,754][44959] Updated weights for policy 1, policy_version 48820 (0.0009) [2023-10-12 22:02:26,124][44959] Updated weights for policy 1, policy_version 48830 (0.0007) [2023-10-12 22:02:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 99745792. Throughput: 0: 1636.0, 1: 1655.3. Samples: 24938366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:02:26,443][43579] Avg episode reward: [(0, '278.190'), (1, '271.190')] [2023-10-12 22:02:29,956][44958] Updated weights for policy 0, policy_version 48580 (0.0008) [2023-10-12 22:02:30,333][44958] Updated weights for policy 0, policy_version 48590 (0.0009) [2023-10-12 22:02:30,489][44959] Updated weights for policy 1, policy_version 48840 (0.0008) [2023-10-12 22:02:30,699][44958] Updated weights for policy 0, policy_version 48600 (0.0008) [2023-10-12 22:02:30,862][44959] Updated weights for policy 1, policy_version 48850 (0.0007) [2023-10-12 22:02:31,226][44959] Updated weights for policy 1, policy_version 48860 (0.0007) [2023-10-12 22:02:31,443][43579] Fps is (10 sec: 13106.6, 60 sec: 13653.2, 300 sec: 13218.3). Total num frames: 99811328. Throughput: 0: 1638.3, 1: 1646.7. Samples: 24956778. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-12 22:02:31,444][43579] Avg episode reward: [(0, '278.810'), (1, '272.330')] [2023-10-12 22:02:34,927][44958] Updated weights for policy 0, policy_version 48610 (0.0007) [2023-10-12 22:02:35,292][44958] Updated weights for policy 0, policy_version 48620 (0.0008) [2023-10-12 22:02:35,309][44959] Updated weights for policy 1, policy_version 48870 (0.0008) [2023-10-12 22:02:35,660][44958] Updated weights for policy 0, policy_version 48630 (0.0008) [2023-10-12 22:02:35,687][44959] Updated weights for policy 1, policy_version 48880 (0.0009) [2023-10-12 22:02:36,032][44958] Updated weights for policy 0, policy_version 48640 (0.0009) [2023-10-12 22:02:36,059][44959] Updated weights for policy 1, policy_version 48890 (0.0008) [2023-10-12 22:02:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 99876864. Throughput: 0: 1637.7, 1: 1660.1. Samples: 24967674. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-12 22:02:36,444][43579] Avg episode reward: [(0, '278.600'), (1, '276.270')] [2023-10-12 22:02:40,044][44959] Updated weights for policy 1, policy_version 48900 (0.0008) [2023-10-12 22:02:40,191][44958] Updated weights for policy 0, policy_version 48650 (0.0009) [2023-10-12 22:02:40,408][44959] Updated weights for policy 1, policy_version 48910 (0.0007) [2023-10-12 22:02:40,556][44958] Updated weights for policy 0, policy_version 48660 (0.0008) [2023-10-12 22:02:40,775][44959] Updated weights for policy 1, policy_version 48920 (0.0008) [2023-10-12 22:02:40,938][44958] Updated weights for policy 0, policy_version 48670 (0.0008) [2023-10-12 22:02:41,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 99942400. Throughput: 0: 1632.5, 1: 1664.3. Samples: 24987566. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-12 22:02:41,444][43579] Avg episode reward: [(0, '274.260'), (1, '270.230')] [2023-10-12 22:02:45,014][44959] Updated weights for policy 1, policy_version 48930 (0.0008) [2023-10-12 22:02:45,381][44958] Updated weights for policy 0, policy_version 48680 (0.0008) [2023-10-12 22:02:45,424][44959] Updated weights for policy 1, policy_version 48940 (0.0008) [2023-10-12 22:02:45,747][44958] Updated weights for policy 0, policy_version 48690 (0.0007) [2023-10-12 22:02:45,795][44959] Updated weights for policy 1, policy_version 48950 (0.0009) [2023-10-12 22:02:46,119][44958] Updated weights for policy 0, policy_version 48700 (0.0007) [2023-10-12 22:02:46,163][44959] Updated weights for policy 1, policy_version 48960 (0.0009) [2023-10-12 22:02:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 100007936. Throughput: 0: 1635.4, 1: 1657.4. Samples: 25006180. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-12 22:02:46,443][43579] Avg episode reward: [(0, '268.740'), (1, '269.640')] [2023-10-12 22:02:50,118][44959] Updated weights for policy 1, policy_version 48970 (0.0007) [2023-10-12 22:02:50,279][44958] Updated weights for policy 0, policy_version 48710 (0.0008) [2023-10-12 22:02:50,495][44959] Updated weights for policy 1, policy_version 48980 (0.0007) [2023-10-12 22:02:50,651][44958] Updated weights for policy 0, policy_version 48720 (0.0008) [2023-10-12 22:02:50,860][44959] Updated weights for policy 1, policy_version 48990 (0.0008) [2023-10-12 22:02:51,019][44958] Updated weights for policy 0, policy_version 48730 (0.0008) [2023-10-12 22:02:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 100073472. Throughput: 0: 1639.9, 1: 1674.1. Samples: 25017450. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-12 22:02:51,443][43579] Avg episode reward: [(0, '272.090'), (1, '274.290')] [2023-10-12 22:02:54,981][44959] Updated weights for policy 1, policy_version 49000 (0.0008) [2023-10-12 22:02:55,329][44958] Updated weights for policy 0, policy_version 48740 (0.0008) [2023-10-12 22:02:55,358][44959] Updated weights for policy 1, policy_version 49010 (0.0008) [2023-10-12 22:02:55,699][44958] Updated weights for policy 0, policy_version 48750 (0.0009) [2023-10-12 22:02:55,722][44959] Updated weights for policy 1, policy_version 49020 (0.0007) [2023-10-12 22:02:56,072][44958] Updated weights for policy 0, policy_version 48760 (0.0008) [2023-10-12 22:02:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 100139008. Throughput: 0: 1641.7, 1: 1650.3. Samples: 25037156. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-12 22:02:56,443][43579] Avg episode reward: [(0, '270.930'), (1, '274.130')] [2023-10-12 22:02:59,743][44959] Updated weights for policy 1, policy_version 49030 (0.0008) [2023-10-12 22:03:00,104][44959] Updated weights for policy 1, policy_version 49040 (0.0009) [2023-10-12 22:03:00,319][44958] Updated weights for policy 0, policy_version 48770 (0.0009) [2023-10-12 22:03:00,481][44959] Updated weights for policy 1, policy_version 49050 (0.0010) [2023-10-12 22:03:00,692][44958] Updated weights for policy 0, policy_version 48780 (0.0008) [2023-10-12 22:03:01,063][44958] Updated weights for policy 0, policy_version 48790 (0.0010) [2023-10-12 22:03:01,436][44958] Updated weights for policy 0, policy_version 48800 (0.0009) [2023-10-12 22:03:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 100204544. Throughput: 0: 1642.1, 1: 1651.9. Samples: 25055562. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-12 22:03:01,444][43579] Avg episode reward: [(0, '268.780'), (1, '272.330')] [2023-10-12 22:03:04,860][44959] Updated weights for policy 1, policy_version 49060 (0.0009) [2023-10-12 22:03:05,227][44959] Updated weights for policy 1, policy_version 49070 (0.0009) [2023-10-12 22:03:05,581][44958] Updated weights for policy 0, policy_version 48810 (0.0009) [2023-10-12 22:03:05,590][44959] Updated weights for policy 1, policy_version 49080 (0.0007) [2023-10-12 22:03:05,960][44958] Updated weights for policy 0, policy_version 48820 (0.0008) [2023-10-12 22:03:06,325][44958] Updated weights for policy 0, policy_version 48830 (0.0009) [2023-10-12 22:03:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 100270080. Throughput: 0: 1633.3, 1: 1655.4. Samples: 25066490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:03:06,444][43579] Avg episode reward: [(0, '265.490'), (1, '270.190')] [2023-10-12 22:03:09,823][44959] Updated weights for policy 1, policy_version 49090 (0.0008) [2023-10-12 22:03:10,184][44959] Updated weights for policy 1, policy_version 49100 (0.0007) [2023-10-12 22:03:10,557][44959] Updated weights for policy 1, policy_version 49110 (0.0008) [2023-10-12 22:03:10,699][44958] Updated weights for policy 0, policy_version 48840 (0.0009) [2023-10-12 22:03:10,920][44959] Updated weights for policy 1, policy_version 49120 (0.0008) [2023-10-12 22:03:11,077][44958] Updated weights for policy 0, policy_version 48850 (0.0010) [2023-10-12 22:03:11,443][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 100302848. Throughput: 0: 1638.2, 1: 1644.6. Samples: 25086092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:03:11,444][43579] Avg episode reward: [(0, '275.340'), (1, '272.960')] [2023-10-12 22:03:11,444][44958] Updated weights for policy 0, policy_version 48860 (0.0009) [2023-10-12 22:03:15,053][44959] Updated weights for policy 1, policy_version 49130 (0.0007) [2023-10-12 22:03:15,409][44959] Updated weights for policy 1, policy_version 49140 (0.0009) [2023-10-12 22:03:15,607][44958] Updated weights for policy 0, policy_version 48870 (0.0007) [2023-10-12 22:03:15,782][44959] Updated weights for policy 1, policy_version 49150 (0.0007) [2023-10-12 22:03:15,986][44958] Updated weights for policy 0, policy_version 48880 (0.0008) [2023-10-12 22:03:16,354][44958] Updated weights for policy 0, policy_version 48890 (0.0007) [2023-10-12 22:03:16,443][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 100368384. Throughput: 0: 1638.3, 1: 1650.2. Samples: 25104760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:03:16,444][43579] Avg episode reward: [(0, '275.120'), (1, '275.030')] [2023-10-12 22:03:19,995][44959] Updated weights for policy 1, policy_version 49160 (0.0010) [2023-10-12 22:03:20,359][44959] Updated weights for policy 1, policy_version 49170 (0.0010) [2023-10-12 22:03:20,432][44958] Updated weights for policy 0, policy_version 48900 (0.0008) [2023-10-12 22:03:20,735][44959] Updated weights for policy 1, policy_version 49180 (0.0008) [2023-10-12 22:03:20,802][44958] Updated weights for policy 0, policy_version 48910 (0.0008) [2023-10-12 22:03:21,178][44958] Updated weights for policy 0, policy_version 48920 (0.0008) [2023-10-12 22:03:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 100433920. Throughput: 0: 1632.2, 1: 1656.6. Samples: 25115672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:03:21,444][43579] Avg episode reward: [(0, '273.100'), (1, '278.960')] [2023-10-12 22:03:24,871][44959] Updated weights for policy 1, policy_version 49190 (0.0007) [2023-10-12 22:03:25,235][44958] Updated weights for policy 0, policy_version 48930 (0.0010) [2023-10-12 22:03:25,240][44959] Updated weights for policy 1, policy_version 49200 (0.0008) [2023-10-12 22:03:25,607][44959] Updated weights for policy 1, policy_version 49210 (0.0008) [2023-10-12 22:03:25,608][44958] Updated weights for policy 0, policy_version 48940 (0.0009) [2023-10-12 22:03:25,979][44958] Updated weights for policy 0, policy_version 48950 (0.0009) [2023-10-12 22:03:26,343][44958] Updated weights for policy 0, policy_version 48960 (0.0007) [2023-10-12 22:03:26,443][43579] Fps is (10 sec: 16384.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 100532224. Throughput: 0: 1638.8, 1: 1645.3. Samples: 25135354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:03:26,444][43579] Avg episode reward: [(0, '267.450'), (1, '275.690')] [2023-10-12 22:03:29,869][44959] Updated weights for policy 1, policy_version 49220 (0.0008) [2023-10-12 22:03:30,234][44959] Updated weights for policy 1, policy_version 49230 (0.0007) [2023-10-12 22:03:30,527][44958] Updated weights for policy 0, policy_version 48970 (0.0009) [2023-10-12 22:03:30,606][44959] Updated weights for policy 1, policy_version 49240 (0.0008) [2023-10-12 22:03:30,898][44958] Updated weights for policy 0, policy_version 48980 (0.0007) [2023-10-12 22:03:31,274][44958] Updated weights for policy 0, policy_version 48990 (0.0007) [2023-10-12 22:03:31,442][43579] Fps is (10 sec: 16384.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 100597760. Throughput: 0: 1640.1, 1: 1643.2. Samples: 25153928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:03:31,443][43579] Avg episode reward: [(0, '269.000'), (1, '278.000')] [2023-10-12 22:03:34,867][44959] Updated weights for policy 1, policy_version 49250 (0.0008) [2023-10-12 22:03:35,162][44958] Updated weights for policy 0, policy_version 49000 (0.0007) [2023-10-12 22:03:35,279][44959] Updated weights for policy 1, policy_version 49260 (0.0009) [2023-10-12 22:03:35,542][44958] Updated weights for policy 0, policy_version 49010 (0.0008) [2023-10-12 22:03:35,653][44959] Updated weights for policy 1, policy_version 49270 (0.0007) [2023-10-12 22:03:35,907][44958] Updated weights for policy 0, policy_version 49020 (0.0008) [2023-10-12 22:03:36,023][44959] Updated weights for policy 1, policy_version 49280 (0.0008) [2023-10-12 22:03:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 100663296. Throughput: 0: 1639.9, 1: 1640.0. Samples: 25165046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:03:36,444][43579] Avg episode reward: [(0, '266.830'), (1, '280.950')] [2023-10-12 22:03:40,067][44959] Updated weights for policy 1, policy_version 49290 (0.0009) [2023-10-12 22:03:40,241][44958] Updated weights for policy 0, policy_version 49030 (0.0008) [2023-10-12 22:03:40,434][44959] Updated weights for policy 1, policy_version 49300 (0.0009) [2023-10-12 22:03:40,628][44958] Updated weights for policy 0, policy_version 49040 (0.0008) [2023-10-12 22:03:40,798][44959] Updated weights for policy 1, policy_version 49310 (0.0008) [2023-10-12 22:03:40,995][44958] Updated weights for policy 0, policy_version 49050 (0.0009) [2023-10-12 22:03:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 100728832. Throughput: 0: 1636.9, 1: 1640.9. Samples: 25184658. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 22:03:41,443][43579] Avg episode reward: [(0, '268.100'), (1, '282.270')] [2023-10-12 22:03:44,952][44959] Updated weights for policy 1, policy_version 49320 (0.0008) [2023-10-12 22:03:45,293][44958] Updated weights for policy 0, policy_version 49060 (0.0009) [2023-10-12 22:03:45,321][44959] Updated weights for policy 1, policy_version 49330 (0.0007) [2023-10-12 22:03:45,666][44958] Updated weights for policy 0, policy_version 49070 (0.0007) [2023-10-12 22:03:45,687][44959] Updated weights for policy 1, policy_version 49340 (0.0008) [2023-10-12 22:03:46,037][44958] Updated weights for policy 0, policy_version 49080 (0.0007) [2023-10-12 22:03:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 100794368. Throughput: 0: 1642.5, 1: 1636.0. Samples: 25203098. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 22:03:46,443][43579] Avg episode reward: [(0, '269.990'), (1, '284.330')] [2023-10-12 22:03:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000049088_50266112.pth... [2023-10-12 22:03:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000049344_50528256.pth... [2023-10-12 22:03:46,491][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000047552_48693248.pth [2023-10-12 22:03:46,493][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000047776_48922624.pth [2023-10-12 22:03:49,866][44959] Updated weights for policy 1, policy_version 49350 (0.0008) [2023-10-12 22:03:50,226][44959] Updated weights for policy 1, policy_version 49360 (0.0008) [2023-10-12 22:03:50,235][44958] Updated weights for policy 0, policy_version 49090 (0.0009) [2023-10-12 22:03:50,595][44959] Updated weights for policy 1, policy_version 49370 (0.0009) [2023-10-12 22:03:50,602][44958] Updated weights for policy 0, policy_version 49100 (0.0008) [2023-10-12 22:03:50,972][44958] Updated weights for policy 0, policy_version 49110 (0.0009) [2023-10-12 22:03:51,341][44958] Updated weights for policy 0, policy_version 49120 (0.0009) [2023-10-12 22:03:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 100859904. Throughput: 0: 1642.0, 1: 1637.7. Samples: 25214080. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 22:03:51,443][43579] Avg episode reward: [(0, '272.390'), (1, '280.780')] [2023-10-12 22:03:54,643][44959] Updated weights for policy 1, policy_version 49380 (0.0009) [2023-10-12 22:03:55,001][44959] Updated weights for policy 1, policy_version 49390 (0.0010) [2023-10-12 22:03:55,374][44959] Updated weights for policy 1, policy_version 49400 (0.0009) [2023-10-12 22:03:55,548][44958] Updated weights for policy 0, policy_version 49130 (0.0007) [2023-10-12 22:03:55,916][44958] Updated weights for policy 0, policy_version 49140 (0.0008) [2023-10-12 22:03:56,276][44958] Updated weights for policy 0, policy_version 49150 (0.0010) [2023-10-12 22:03:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 100925440. Throughput: 0: 1643.1, 1: 1643.6. Samples: 25233994. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 22:03:56,443][43579] Avg episode reward: [(0, '276.400'), (1, '285.260')] [2023-10-12 22:03:59,371][44959] Updated weights for policy 1, policy_version 49410 (0.0008) [2023-10-12 22:03:59,730][44959] Updated weights for policy 1, policy_version 49420 (0.0008) [2023-10-12 22:04:00,100][44959] Updated weights for policy 1, policy_version 49430 (0.0009) [2023-10-12 22:04:00,304][44958] Updated weights for policy 0, policy_version 49160 (0.0008) [2023-10-12 22:04:00,465][44959] Updated weights for policy 1, policy_version 49440 (0.0007) [2023-10-12 22:04:00,662][44958] Updated weights for policy 0, policy_version 49170 (0.0009) [2023-10-12 22:04:01,032][44958] Updated weights for policy 0, policy_version 49180 (0.0008) [2023-10-12 22:04:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 100990976. Throughput: 0: 1642.5, 1: 1647.3. Samples: 25252798. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 22:04:01,443][43579] Avg episode reward: [(0, '273.800'), (1, '285.630')] [2023-10-12 22:04:04,684][44959] Updated weights for policy 1, policy_version 49450 (0.0007) [2023-10-12 22:04:05,046][44959] Updated weights for policy 1, policy_version 49460 (0.0009) [2023-10-12 22:04:05,141][44958] Updated weights for policy 0, policy_version 49190 (0.0007) [2023-10-12 22:04:05,412][44959] Updated weights for policy 1, policy_version 49470 (0.0009) [2023-10-12 22:04:05,517][44958] Updated weights for policy 0, policy_version 49200 (0.0008) [2023-10-12 22:04:05,889][44958] Updated weights for policy 0, policy_version 49210 (0.0008) [2023-10-12 22:04:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101056512. Throughput: 0: 1654.0, 1: 1647.5. Samples: 25264242. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 22:04:06,443][43579] Avg episode reward: [(0, '279.190'), (1, '283.060')] [2023-10-12 22:04:09,587][44959] Updated weights for policy 1, policy_version 49480 (0.0010) [2023-10-12 22:04:09,855][44958] Updated weights for policy 0, policy_version 49220 (0.0008) [2023-10-12 22:04:09,954][44959] Updated weights for policy 1, policy_version 49490 (0.0007) [2023-10-12 22:04:10,228][44958] Updated weights for policy 0, policy_version 49230 (0.0007) [2023-10-12 22:04:10,319][44959] Updated weights for policy 1, policy_version 49500 (0.0007) [2023-10-12 22:04:10,607][44958] Updated weights for policy 0, policy_version 49240 (0.0008) [2023-10-12 22:04:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 101122048. Throughput: 0: 1648.5, 1: 1641.0. Samples: 25283384. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 22:04:11,443][43579] Avg episode reward: [(0, '278.740'), (1, '282.240')] [2023-10-12 22:04:14,336][44959] Updated weights for policy 1, policy_version 49510 (0.0009) [2023-10-12 22:04:14,708][44959] Updated weights for policy 1, policy_version 49520 (0.0010) [2023-10-12 22:04:14,821][44958] Updated weights for policy 0, policy_version 49250 (0.0010) [2023-10-12 22:04:15,067][44959] Updated weights for policy 1, policy_version 49530 (0.0008) [2023-10-12 22:04:15,192][44958] Updated weights for policy 0, policy_version 49260 (0.0009) [2023-10-12 22:04:15,558][44958] Updated weights for policy 0, policy_version 49270 (0.0009) [2023-10-12 22:04:15,931][44958] Updated weights for policy 0, policy_version 49280 (0.0010) [2023-10-12 22:04:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 101187584. Throughput: 0: 1647.8, 1: 1655.8. Samples: 25302590. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:04:16,444][43579] Avg episode reward: [(0, '277.090'), (1, '279.250')] [2023-10-12 22:04:19,253][44959] Updated weights for policy 1, policy_version 49540 (0.0007) [2023-10-12 22:04:19,621][44959] Updated weights for policy 1, policy_version 49550 (0.0007) [2023-10-12 22:04:19,988][44959] Updated weights for policy 1, policy_version 49560 (0.0008) [2023-10-12 22:04:20,246][44958] Updated weights for policy 0, policy_version 49290 (0.0008) [2023-10-12 22:04:20,614][44958] Updated weights for policy 0, policy_version 49300 (0.0009) [2023-10-12 22:04:20,988][44958] Updated weights for policy 0, policy_version 49310 (0.0010) [2023-10-12 22:04:21,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 101253120. Throughput: 0: 1649.0, 1: 1657.2. Samples: 25313824. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:04:21,444][43579] Avg episode reward: [(0, '274.790'), (1, '278.670')] [2023-10-12 22:04:24,140][44959] Updated weights for policy 1, policy_version 49570 (0.0007) [2023-10-12 22:04:24,507][44959] Updated weights for policy 1, policy_version 49580 (0.0007) [2023-10-12 22:04:24,867][44959] Updated weights for policy 1, policy_version 49590 (0.0007) [2023-10-12 22:04:25,196][44958] Updated weights for policy 0, policy_version 49320 (0.0009) [2023-10-12 22:04:25,235][44959] Updated weights for policy 1, policy_version 49600 (0.0007) [2023-10-12 22:04:25,556][44958] Updated weights for policy 0, policy_version 49330 (0.0007) [2023-10-12 22:04:25,931][44958] Updated weights for policy 0, policy_version 49340 (0.0008) [2023-10-12 22:04:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 101318656. Throughput: 0: 1642.3, 1: 1644.9. Samples: 25332580. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:04:26,443][43579] Avg episode reward: [(0, '275.740'), (1, '277.150')] [2023-10-12 22:04:29,529][44959] Updated weights for policy 1, policy_version 49610 (0.0010) [2023-10-12 22:04:29,902][44959] Updated weights for policy 1, policy_version 49620 (0.0010) [2023-10-12 22:04:30,102][44958] Updated weights for policy 0, policy_version 49350 (0.0009) [2023-10-12 22:04:30,271][44959] Updated weights for policy 1, policy_version 49630 (0.0008) [2023-10-12 22:04:30,473][44958] Updated weights for policy 0, policy_version 49360 (0.0008) [2023-10-12 22:04:30,848][44958] Updated weights for policy 0, policy_version 49370 (0.0010) [2023-10-12 22:04:31,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101384192. Throughput: 0: 1640.1, 1: 1652.2. Samples: 25351252. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:04:31,443][43579] Avg episode reward: [(0, '277.900'), (1, '273.520')] [2023-10-12 22:04:34,530][44959] Updated weights for policy 1, policy_version 49640 (0.0008) [2023-10-12 22:04:34,896][44959] Updated weights for policy 1, policy_version 49650 (0.0008) [2023-10-12 22:04:35,147][44958] Updated weights for policy 0, policy_version 49380 (0.0008) [2023-10-12 22:04:35,258][44959] Updated weights for policy 1, policy_version 49660 (0.0008) [2023-10-12 22:04:35,522][44958] Updated weights for policy 0, policy_version 49390 (0.0009) [2023-10-12 22:04:35,904][44958] Updated weights for policy 0, policy_version 49400 (0.0007) [2023-10-12 22:04:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101449728. Throughput: 0: 1644.0, 1: 1650.8. Samples: 25362344. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:04:36,444][43579] Avg episode reward: [(0, '281.170'), (1, '275.920')] [2023-10-12 22:04:39,446][44959] Updated weights for policy 1, policy_version 49670 (0.0007) [2023-10-12 22:04:39,815][44959] Updated weights for policy 1, policy_version 49680 (0.0008) [2023-10-12 22:04:39,902][44958] Updated weights for policy 0, policy_version 49410 (0.0008) [2023-10-12 22:04:40,180][44959] Updated weights for policy 1, policy_version 49690 (0.0010) [2023-10-12 22:04:40,268][44958] Updated weights for policy 0, policy_version 49420 (0.0010) [2023-10-12 22:04:40,639][44958] Updated weights for policy 0, policy_version 49430 (0.0010) [2023-10-12 22:04:41,014][44958] Updated weights for policy 0, policy_version 49440 (0.0007) [2023-10-12 22:04:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101515264. Throughput: 0: 1635.8, 1: 1640.0. Samples: 25381404. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:04:41,443][43579] Avg episode reward: [(0, '281.250'), (1, '271.690')] [2023-10-12 22:04:44,318][44959] Updated weights for policy 1, policy_version 49700 (0.0007) [2023-10-12 22:04:44,688][44959] Updated weights for policy 1, policy_version 49710 (0.0008) [2023-10-12 22:04:45,048][44958] Updated weights for policy 0, policy_version 49450 (0.0007) [2023-10-12 22:04:45,049][44959] Updated weights for policy 1, policy_version 49720 (0.0008) [2023-10-12 22:04:45,428][44958] Updated weights for policy 0, policy_version 49460 (0.0009) [2023-10-12 22:04:45,805][44958] Updated weights for policy 0, policy_version 49470 (0.0008) [2023-10-12 22:04:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101580800. Throughput: 0: 1639.9, 1: 1645.6. Samples: 25400644. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:04:46,444][43579] Avg episode reward: [(0, '281.280'), (1, '270.440')] [2023-10-12 22:04:49,220][44959] Updated weights for policy 1, policy_version 49730 (0.0008) [2023-10-12 22:04:49,599][44959] Updated weights for policy 1, policy_version 49740 (0.0009) [2023-10-12 22:04:49,971][44959] Updated weights for policy 1, policy_version 49750 (0.0009) [2023-10-12 22:04:50,109][44958] Updated weights for policy 0, policy_version 49480 (0.0009) [2023-10-12 22:04:50,333][44959] Updated weights for policy 1, policy_version 49760 (0.0009) [2023-10-12 22:04:50,486][44958] Updated weights for policy 0, policy_version 49490 (0.0008) [2023-10-12 22:04:50,859][44958] Updated weights for policy 0, policy_version 49500 (0.0009) [2023-10-12 22:04:51,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101646336. Throughput: 0: 1635.6, 1: 1645.9. Samples: 25411906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:04:51,444][43579] Avg episode reward: [(0, '284.390'), (1, '266.430')] [2023-10-12 22:04:54,703][44959] Updated weights for policy 1, policy_version 49770 (0.0007) [2023-10-12 22:04:54,751][44958] Updated weights for policy 0, policy_version 49510 (0.0007) [2023-10-12 22:04:55,079][44959] Updated weights for policy 1, policy_version 49780 (0.0007) [2023-10-12 22:04:55,128][44958] Updated weights for policy 0, policy_version 49520 (0.0007) [2023-10-12 22:04:55,443][44959] Updated weights for policy 1, policy_version 49790 (0.0008) [2023-10-12 22:04:55,503][44958] Updated weights for policy 0, policy_version 49530 (0.0007) [2023-10-12 22:04:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101711872. Throughput: 0: 1634.0, 1: 1647.4. Samples: 25431048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:04:56,443][43579] Avg episode reward: [(0, '286.140'), (1, '268.990')] [2023-10-12 22:04:59,557][44959] Updated weights for policy 1, policy_version 49800 (0.0009) [2023-10-12 22:04:59,836][44958] Updated weights for policy 0, policy_version 49540 (0.0008) [2023-10-12 22:04:59,930][44959] Updated weights for policy 1, policy_version 49810 (0.0007) [2023-10-12 22:05:00,208][44958] Updated weights for policy 0, policy_version 49550 (0.0008) [2023-10-12 22:05:00,284][44959] Updated weights for policy 1, policy_version 49820 (0.0007) [2023-10-12 22:05:00,582][44958] Updated weights for policy 0, policy_version 49560 (0.0009) [2023-10-12 22:05:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 101777408. Throughput: 0: 1637.6, 1: 1639.4. Samples: 25450052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:05:01,444][43579] Avg episode reward: [(0, '283.410'), (1, '269.340')] [2023-10-12 22:05:04,564][44959] Updated weights for policy 1, policy_version 49830 (0.0009) [2023-10-12 22:05:04,718][44958] Updated weights for policy 0, policy_version 49570 (0.0007) [2023-10-12 22:05:04,951][44959] Updated weights for policy 1, policy_version 49840 (0.0007) [2023-10-12 22:05:05,092][44958] Updated weights for policy 0, policy_version 49580 (0.0008) [2023-10-12 22:05:05,310][44959] Updated weights for policy 1, policy_version 49850 (0.0007) [2023-10-12 22:05:05,473][44958] Updated weights for policy 0, policy_version 49590 (0.0009) [2023-10-12 22:05:05,849][44958] Updated weights for policy 0, policy_version 49600 (0.0008) [2023-10-12 22:05:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101842944. Throughput: 0: 1640.6, 1: 1639.9. Samples: 25461448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:05:06,443][43579] Avg episode reward: [(0, '283.320'), (1, '270.960')] [2023-10-12 22:05:09,482][44959] Updated weights for policy 1, policy_version 49860 (0.0008) [2023-10-12 22:05:09,854][44959] Updated weights for policy 1, policy_version 49870 (0.0009) [2023-10-12 22:05:09,940][44958] Updated weights for policy 0, policy_version 49610 (0.0009) [2023-10-12 22:05:10,222][44959] Updated weights for policy 1, policy_version 49880 (0.0007) [2023-10-12 22:05:10,306][44958] Updated weights for policy 0, policy_version 49620 (0.0009) [2023-10-12 22:05:10,682][44958] Updated weights for policy 0, policy_version 49630 (0.0008) [2023-10-12 22:05:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101908480. Throughput: 0: 1636.8, 1: 1642.4. Samples: 25480140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:05:11,443][43579] Avg episode reward: [(0, '284.840'), (1, '275.590')] [2023-10-12 22:05:14,463][44959] Updated weights for policy 1, policy_version 49890 (0.0007) [2023-10-12 22:05:14,824][44959] Updated weights for policy 1, policy_version 49900 (0.0008) [2023-10-12 22:05:14,942][44958] Updated weights for policy 0, policy_version 49640 (0.0008) [2023-10-12 22:05:15,190][44959] Updated weights for policy 1, policy_version 49910 (0.0009) [2023-10-12 22:05:15,308][44958] Updated weights for policy 0, policy_version 49650 (0.0010) [2023-10-12 22:05:15,555][44959] Updated weights for policy 1, policy_version 49920 (0.0007) [2023-10-12 22:05:15,682][44958] Updated weights for policy 0, policy_version 49660 (0.0009) [2023-10-12 22:05:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 101974016. Throughput: 0: 1639.6, 1: 1644.3. Samples: 25499030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:05:16,443][43579] Avg episode reward: [(0, '282.140'), (1, '278.490')] [2023-10-12 22:05:19,679][44959] Updated weights for policy 1, policy_version 49930 (0.0008) [2023-10-12 22:05:19,909][44958] Updated weights for policy 0, policy_version 49670 (0.0008) [2023-10-12 22:05:20,056][44959] Updated weights for policy 1, policy_version 49940 (0.0010) [2023-10-12 22:05:20,270][44958] Updated weights for policy 0, policy_version 49680 (0.0009) [2023-10-12 22:05:20,420][44959] Updated weights for policy 1, policy_version 49950 (0.0007) [2023-10-12 22:05:20,641][44958] Updated weights for policy 0, policy_version 49690 (0.0009) [2023-10-12 22:05:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102039552. Throughput: 0: 1644.2, 1: 1648.0. Samples: 25510496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:05:21,444][43579] Avg episode reward: [(0, '279.450'), (1, '282.840')] [2023-10-12 22:05:24,549][44959] Updated weights for policy 1, policy_version 49960 (0.0008) [2023-10-12 22:05:24,555][44958] Updated weights for policy 0, policy_version 49700 (0.0009) [2023-10-12 22:05:24,909][44959] Updated weights for policy 1, policy_version 49970 (0.0009) [2023-10-12 22:05:24,917][44958] Updated weights for policy 0, policy_version 49710 (0.0009) [2023-10-12 22:05:25,283][44959] Updated weights for policy 1, policy_version 49980 (0.0009) [2023-10-12 22:05:25,291][44958] Updated weights for policy 0, policy_version 49720 (0.0008) [2023-10-12 22:05:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102105088. Throughput: 0: 1642.3, 1: 1645.5. Samples: 25529354. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 22:05:26,444][43579] Avg episode reward: [(0, '279.520'), (1, '278.520')] [2023-10-12 22:05:29,433][44959] Updated weights for policy 1, policy_version 49990 (0.0008) [2023-10-12 22:05:29,520][44958] Updated weights for policy 0, policy_version 49730 (0.0009) [2023-10-12 22:05:29,799][44959] Updated weights for policy 1, policy_version 50000 (0.0007) [2023-10-12 22:05:29,897][44958] Updated weights for policy 0, policy_version 49740 (0.0009) [2023-10-12 22:05:30,178][44959] Updated weights for policy 1, policy_version 50010 (0.0009) [2023-10-12 22:05:30,284][44958] Updated weights for policy 0, policy_version 49750 (0.0009) [2023-10-12 22:05:30,663][44958] Updated weights for policy 0, policy_version 49760 (0.0008) [2023-10-12 22:05:31,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102170624. Throughput: 0: 1645.6, 1: 1645.0. Samples: 25548724. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 22:05:31,443][43579] Avg episode reward: [(0, '275.760'), (1, '280.860')] [2023-10-12 22:05:34,291][44959] Updated weights for policy 1, policy_version 50020 (0.0008) [2023-10-12 22:05:34,664][44959] Updated weights for policy 1, policy_version 50030 (0.0008) [2023-10-12 22:05:35,026][44959] Updated weights for policy 1, policy_version 50040 (0.0008) [2023-10-12 22:05:35,048][44958] Updated weights for policy 0, policy_version 49770 (0.0007) [2023-10-12 22:05:35,417][44958] Updated weights for policy 0, policy_version 49780 (0.0008) [2023-10-12 22:05:35,799][44958] Updated weights for policy 0, policy_version 49790 (0.0008) [2023-10-12 22:05:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102236160. Throughput: 0: 1647.4, 1: 1642.6. Samples: 25559958. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 22:05:36,443][43579] Avg episode reward: [(0, '272.150'), (1, '280.620')] [2023-10-12 22:05:39,039][44959] Updated weights for policy 1, policy_version 50050 (0.0008) [2023-10-12 22:05:39,407][44959] Updated weights for policy 1, policy_version 50060 (0.0009) [2023-10-12 22:05:39,780][44959] Updated weights for policy 1, policy_version 50070 (0.0009) [2023-10-12 22:05:40,142][44958] Updated weights for policy 0, policy_version 49800 (0.0009) [2023-10-12 22:05:40,144][44959] Updated weights for policy 1, policy_version 50080 (0.0007) [2023-10-12 22:05:40,516][44958] Updated weights for policy 0, policy_version 49810 (0.0007) [2023-10-12 22:05:40,887][44958] Updated weights for policy 0, policy_version 49820 (0.0007) [2023-10-12 22:05:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 102301696. Throughput: 0: 1646.9, 1: 1634.1. Samples: 25578694. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 22:05:41,444][43579] Avg episode reward: [(0, '271.040'), (1, '283.140')] [2023-10-12 22:05:44,302][44959] Updated weights for policy 1, policy_version 50090 (0.0010) [2023-10-12 22:05:44,655][44959] Updated weights for policy 1, policy_version 50100 (0.0011) [2023-10-12 22:05:45,003][44958] Updated weights for policy 0, policy_version 49830 (0.0008) [2023-10-12 22:05:45,025][44959] Updated weights for policy 1, policy_version 50110 (0.0008) [2023-10-12 22:05:45,377][44958] Updated weights for policy 0, policy_version 49840 (0.0009) [2023-10-12 22:05:45,749][44958] Updated weights for policy 0, policy_version 49850 (0.0010) [2023-10-12 22:05:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102367232. Throughput: 0: 1642.5, 1: 1650.5. Samples: 25598238. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 22:05:46,443][43579] Avg episode reward: [(0, '264.980'), (1, '283.410')] [2023-10-12 22:05:46,450][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000050112_51314688.pth... [2023-10-12 22:05:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000049856_51052544.pth... [2023-10-12 22:05:46,483][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000048320_49479680.pth [2023-10-12 22:05:46,484][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000048576_49741824.pth [2023-10-12 22:05:49,341][44959] Updated weights for policy 1, policy_version 50120 (0.0007) [2023-10-12 22:05:49,721][44959] Updated weights for policy 1, policy_version 50130 (0.0009) [2023-10-12 22:05:50,022][44958] Updated weights for policy 0, policy_version 49860 (0.0008) [2023-10-12 22:05:50,096][44959] Updated weights for policy 1, policy_version 50140 (0.0010) [2023-10-12 22:05:50,394][44958] Updated weights for policy 0, policy_version 49870 (0.0007) [2023-10-12 22:05:50,763][44958] Updated weights for policy 0, policy_version 49880 (0.0007) [2023-10-12 22:05:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102432768. Throughput: 0: 1638.7, 1: 1646.3. Samples: 25609274. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 22:05:51,444][43579] Avg episode reward: [(0, '268.030'), (1, '285.080')] [2023-10-12 22:05:54,339][44959] Updated weights for policy 1, policy_version 50150 (0.0008) [2023-10-12 22:05:54,712][44959] Updated weights for policy 1, policy_version 50160 (0.0007) [2023-10-12 22:05:54,828][44958] Updated weights for policy 0, policy_version 49890 (0.0010) [2023-10-12 22:05:55,074][44959] Updated weights for policy 1, policy_version 50170 (0.0007) [2023-10-12 22:05:55,195][44958] Updated weights for policy 0, policy_version 49900 (0.0008) [2023-10-12 22:05:55,564][44958] Updated weights for policy 0, policy_version 49910 (0.0007) [2023-10-12 22:05:55,931][44958] Updated weights for policy 0, policy_version 49920 (0.0007) [2023-10-12 22:05:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102498304. Throughput: 0: 1642.7, 1: 1641.1. Samples: 25627912. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-12 22:05:56,444][43579] Avg episode reward: [(0, '267.860'), (1, '284.760')] [2023-10-12 22:05:59,040][44959] Updated weights for policy 1, policy_version 50180 (0.0007) [2023-10-12 22:05:59,410][44959] Updated weights for policy 1, policy_version 50190 (0.0009) [2023-10-12 22:05:59,775][44959] Updated weights for policy 1, policy_version 50200 (0.0009) [2023-10-12 22:06:00,121][44958] Updated weights for policy 0, policy_version 49930 (0.0007) [2023-10-12 22:06:00,485][44958] Updated weights for policy 0, policy_version 49940 (0.0008) [2023-10-12 22:06:00,854][44958] Updated weights for policy 0, policy_version 49950 (0.0009) [2023-10-12 22:06:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102563840. Throughput: 0: 1640.5, 1: 1651.0. Samples: 25647148. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-12 22:06:01,444][43579] Avg episode reward: [(0, '264.480'), (1, '284.210')] [2023-10-12 22:06:03,842][44959] Updated weights for policy 1, policy_version 50210 (0.0008) [2023-10-12 22:06:04,213][44959] Updated weights for policy 1, policy_version 50220 (0.0008) [2023-10-12 22:06:04,574][44959] Updated weights for policy 1, policy_version 50230 (0.0008) [2023-10-12 22:06:04,947][44959] Updated weights for policy 1, policy_version 50240 (0.0010) [2023-10-12 22:06:04,997][44958] Updated weights for policy 0, policy_version 49960 (0.0007) [2023-10-12 22:06:05,371][44958] Updated weights for policy 0, policy_version 49970 (0.0009) [2023-10-12 22:06:05,734][44958] Updated weights for policy 0, policy_version 49980 (0.0009) [2023-10-12 22:06:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102629376. Throughput: 0: 1636.4, 1: 1645.3. Samples: 25658172. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-12 22:06:06,444][43579] Avg episode reward: [(0, '264.010'), (1, '283.400')] [2023-10-12 22:06:09,175][44959] Updated weights for policy 1, policy_version 50250 (0.0011) [2023-10-12 22:06:09,541][44959] Updated weights for policy 1, policy_version 50260 (0.0009) [2023-10-12 22:06:09,906][44958] Updated weights for policy 0, policy_version 49990 (0.0009) [2023-10-12 22:06:09,908][44959] Updated weights for policy 1, policy_version 50270 (0.0009) [2023-10-12 22:06:10,276][44958] Updated weights for policy 0, policy_version 50000 (0.0010) [2023-10-12 22:06:10,639][44958] Updated weights for policy 0, policy_version 50010 (0.0007) [2023-10-12 22:06:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102694912. Throughput: 0: 1640.2, 1: 1643.6. Samples: 25677126. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-12 22:06:11,443][43579] Avg episode reward: [(0, '267.290'), (1, '282.680')] [2023-10-12 22:06:14,066][44959] Updated weights for policy 1, policy_version 50280 (0.0007) [2023-10-12 22:06:14,434][44959] Updated weights for policy 1, policy_version 50290 (0.0008) [2023-10-12 22:06:14,719][44958] Updated weights for policy 0, policy_version 50020 (0.0008) [2023-10-12 22:06:14,810][44959] Updated weights for policy 1, policy_version 50300 (0.0008) [2023-10-12 22:06:15,084][44958] Updated weights for policy 0, policy_version 50030 (0.0007) [2023-10-12 22:06:15,461][44958] Updated weights for policy 0, policy_version 50040 (0.0010) [2023-10-12 22:06:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102760448. Throughput: 0: 1633.1, 1: 1654.9. Samples: 25696684. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-12 22:06:16,444][43579] Avg episode reward: [(0, '271.690'), (1, '277.160')] [2023-10-12 22:06:19,021][44959] Updated weights for policy 1, policy_version 50310 (0.0008) [2023-10-12 22:06:19,389][44959] Updated weights for policy 1, policy_version 50320 (0.0009) [2023-10-12 22:06:19,753][44959] Updated weights for policy 1, policy_version 50330 (0.0009) [2023-10-12 22:06:19,869][44958] Updated weights for policy 0, policy_version 50050 (0.0009) [2023-10-12 22:06:20,275][44958] Updated weights for policy 0, policy_version 50060 (0.0009) [2023-10-12 22:06:20,651][44958] Updated weights for policy 0, policy_version 50070 (0.0012) [2023-10-12 22:06:21,029][44958] Updated weights for policy 0, policy_version 50080 (0.0012) [2023-10-12 22:06:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102825984. Throughput: 0: 1631.9, 1: 1650.4. Samples: 25707664. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-12 22:06:21,444][43579] Avg episode reward: [(0, '270.650'), (1, '279.740')] [2023-10-12 22:06:23,853][44959] Updated weights for policy 1, policy_version 50340 (0.0007) [2023-10-12 22:06:24,220][44959] Updated weights for policy 1, policy_version 50350 (0.0007) [2023-10-12 22:06:24,582][44959] Updated weights for policy 1, policy_version 50360 (0.0007) [2023-10-12 22:06:25,110][44958] Updated weights for policy 0, policy_version 50090 (0.0008) [2023-10-12 22:06:25,482][44958] Updated weights for policy 0, policy_version 50100 (0.0009) [2023-10-12 22:06:25,856][44958] Updated weights for policy 0, policy_version 50110 (0.0009) [2023-10-12 22:06:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102891520. Throughput: 0: 1631.3, 1: 1650.9. Samples: 25726390. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-12 22:06:26,443][43579] Avg episode reward: [(0, '274.070'), (1, '275.980')] [2023-10-12 22:06:28,767][44959] Updated weights for policy 1, policy_version 50370 (0.0008) [2023-10-12 22:06:29,140][44959] Updated weights for policy 1, policy_version 50380 (0.0011) [2023-10-12 22:06:29,497][44959] Updated weights for policy 1, policy_version 50390 (0.0009) [2023-10-12 22:06:29,864][44959] Updated weights for policy 1, policy_version 50400 (0.0009) [2023-10-12 22:06:30,026][44958] Updated weights for policy 0, policy_version 50120 (0.0008) [2023-10-12 22:06:30,400][44958] Updated weights for policy 0, policy_version 50130 (0.0009) [2023-10-12 22:06:30,787][44958] Updated weights for policy 0, policy_version 50140 (0.0009) [2023-10-12 22:06:31,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 102957056. Throughput: 0: 1629.9, 1: 1650.5. Samples: 25745858. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-12 22:06:31,443][43579] Avg episode reward: [(0, '272.820'), (1, '272.970')] [2023-10-12 22:06:33,910][44959] Updated weights for policy 1, policy_version 50410 (0.0010) [2023-10-12 22:06:34,266][44959] Updated weights for policy 1, policy_version 50420 (0.0009) [2023-10-12 22:06:34,626][44959] Updated weights for policy 1, policy_version 50430 (0.0010) [2023-10-12 22:06:34,982][44958] Updated weights for policy 0, policy_version 50150 (0.0010) [2023-10-12 22:06:35,353][44958] Updated weights for policy 0, policy_version 50160 (0.0008) [2023-10-12 22:06:35,730][44958] Updated weights for policy 0, policy_version 50170 (0.0008) [2023-10-12 22:06:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 103022592. Throughput: 0: 1633.8, 1: 1642.4. Samples: 25756704. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-12 22:06:36,443][43579] Avg episode reward: [(0, '273.390'), (1, '273.470')] [2023-10-12 22:06:38,894][44959] Updated weights for policy 1, policy_version 50440 (0.0008) [2023-10-12 22:06:39,268][44959] Updated weights for policy 1, policy_version 50450 (0.0008) [2023-10-12 22:06:39,637][44959] Updated weights for policy 1, policy_version 50460 (0.0010) [2023-10-12 22:06:39,883][44958] Updated weights for policy 0, policy_version 50180 (0.0008) [2023-10-12 22:06:40,259][44958] Updated weights for policy 0, policy_version 50190 (0.0008) [2023-10-12 22:06:40,640][44958] Updated weights for policy 0, policy_version 50200 (0.0010) [2023-10-12 22:06:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 103088128. Throughput: 0: 1640.4, 1: 1650.6. Samples: 25776006. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-12 22:06:41,444][43579] Avg episode reward: [(0, '274.470'), (1, '275.420')] [2023-10-12 22:06:43,760][44959] Updated weights for policy 1, policy_version 50470 (0.0010) [2023-10-12 22:06:44,133][44959] Updated weights for policy 1, policy_version 50480 (0.0009) [2023-10-12 22:06:44,500][44959] Updated weights for policy 1, policy_version 50490 (0.0008) [2023-10-12 22:06:44,745][44958] Updated weights for policy 0, policy_version 50210 (0.0008) [2023-10-12 22:06:45,113][44958] Updated weights for policy 0, policy_version 50220 (0.0009) [2023-10-12 22:06:45,477][44958] Updated weights for policy 0, policy_version 50230 (0.0010) [2023-10-12 22:06:45,847][44958] Updated weights for policy 0, policy_version 50240 (0.0009) [2023-10-12 22:06:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 103153664. Throughput: 0: 1641.0, 1: 1655.7. Samples: 25795500. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-12 22:06:46,443][43579] Avg episode reward: [(0, '273.670'), (1, '279.170')] [2023-10-12 22:06:48,594][44959] Updated weights for policy 1, policy_version 50500 (0.0009) [2023-10-12 22:06:48,960][44959] Updated weights for policy 1, policy_version 50510 (0.0008) [2023-10-12 22:06:49,332][44959] Updated weights for policy 1, policy_version 50520 (0.0007) [2023-10-12 22:06:50,121][44958] Updated weights for policy 0, policy_version 50250 (0.0008) [2023-10-12 22:06:50,498][44958] Updated weights for policy 0, policy_version 50260 (0.0008) [2023-10-12 22:06:50,880][44958] Updated weights for policy 0, policy_version 50270 (0.0007) [2023-10-12 22:06:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 103219200. Throughput: 0: 1641.2, 1: 1649.8. Samples: 25806264. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-12 22:06:51,444][43579] Avg episode reward: [(0, '276.060'), (1, '280.420')] [2023-10-12 22:06:53,449][44959] Updated weights for policy 1, policy_version 50530 (0.0009) [2023-10-12 22:06:53,818][44959] Updated weights for policy 1, policy_version 50540 (0.0007) [2023-10-12 22:06:54,190][44959] Updated weights for policy 1, policy_version 50550 (0.0007) [2023-10-12 22:06:54,561][44959] Updated weights for policy 1, policy_version 50560 (0.0008) [2023-10-12 22:06:55,004][44958] Updated weights for policy 0, policy_version 50280 (0.0010) [2023-10-12 22:06:55,381][44958] Updated weights for policy 0, policy_version 50290 (0.0010) [2023-10-12 22:06:55,749][44958] Updated weights for policy 0, policy_version 50300 (0.0009) [2023-10-12 22:06:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 103284736. Throughput: 0: 1641.2, 1: 1660.3. Samples: 25825694. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-12 22:06:56,444][43579] Avg episode reward: [(0, '279.480'), (1, '285.350')] [2023-10-12 22:06:58,727][44959] Updated weights for policy 1, policy_version 50570 (0.0009) [2023-10-12 22:06:59,104][44959] Updated weights for policy 1, policy_version 50580 (0.0009) [2023-10-12 22:06:59,470][44959] Updated weights for policy 1, policy_version 50590 (0.0008) [2023-10-12 22:06:59,896][44958] Updated weights for policy 0, policy_version 50310 (0.0008) [2023-10-12 22:07:00,276][44958] Updated weights for policy 0, policy_version 50320 (0.0010) [2023-10-12 22:07:00,658][44958] Updated weights for policy 0, policy_version 50330 (0.0009) [2023-10-12 22:07:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 103350272. Throughput: 0: 1641.0, 1: 1654.1. Samples: 25844966. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-12 22:07:01,443][43579] Avg episode reward: [(0, '281.560'), (1, '290.050')] [2023-10-12 22:07:03,885][44959] Updated weights for policy 1, policy_version 50600 (0.0007) [2023-10-12 22:07:04,261][44959] Updated weights for policy 1, policy_version 50610 (0.0009) [2023-10-12 22:07:04,637][44959] Updated weights for policy 1, policy_version 50620 (0.0011) [2023-10-12 22:07:05,033][44958] Updated weights for policy 0, policy_version 50340 (0.0009) [2023-10-12 22:07:05,418][44958] Updated weights for policy 0, policy_version 50350 (0.0011) [2023-10-12 22:07:05,794][44958] Updated weights for policy 0, policy_version 50360 (0.0009) [2023-10-12 22:07:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 103415808. Throughput: 0: 1640.7, 1: 1648.2. Samples: 25855664. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-12 22:07:06,443][43579] Avg episode reward: [(0, '280.090'), (1, '284.070')] [2023-10-12 22:07:08,593][44959] Updated weights for policy 1, policy_version 50630 (0.0010) [2023-10-12 22:07:08,962][44959] Updated weights for policy 1, policy_version 50640 (0.0010) [2023-10-12 22:07:09,334][44959] Updated weights for policy 1, policy_version 50650 (0.0010) [2023-10-12 22:07:09,773][44958] Updated weights for policy 0, policy_version 50370 (0.0008) [2023-10-12 22:07:10,150][44958] Updated weights for policy 0, policy_version 50380 (0.0008) [2023-10-12 22:07:10,516][44958] Updated weights for policy 0, policy_version 50390 (0.0009) [2023-10-12 22:07:10,889][44958] Updated weights for policy 0, policy_version 50400 (0.0008) [2023-10-12 22:07:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 103481344. Throughput: 0: 1645.4, 1: 1654.9. Samples: 25874904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:11,443][43579] Avg episode reward: [(0, '278.710'), (1, '279.140')] [2023-10-12 22:07:13,509][44959] Updated weights for policy 1, policy_version 50660 (0.0008) [2023-10-12 22:07:13,884][44959] Updated weights for policy 1, policy_version 50670 (0.0008) [2023-10-12 22:07:14,241][44959] Updated weights for policy 1, policy_version 50680 (0.0011) [2023-10-12 22:07:15,033][44958] Updated weights for policy 0, policy_version 50410 (0.0009) [2023-10-12 22:07:15,399][44958] Updated weights for policy 0, policy_version 50420 (0.0010) [2023-10-12 22:07:15,772][44958] Updated weights for policy 0, policy_version 50430 (0.0009) [2023-10-12 22:07:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 103546880. Throughput: 0: 1648.6, 1: 1653.2. Samples: 25894438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:16,443][43579] Avg episode reward: [(0, '278.070'), (1, '277.110')] [2023-10-12 22:07:18,236][44959] Updated weights for policy 1, policy_version 50690 (0.0010) [2023-10-12 22:07:18,603][44959] Updated weights for policy 1, policy_version 50700 (0.0008) [2023-10-12 22:07:18,974][44959] Updated weights for policy 1, policy_version 50710 (0.0010) [2023-10-12 22:07:19,337][44959] Updated weights for policy 1, policy_version 50720 (0.0010) [2023-10-12 22:07:19,975][44958] Updated weights for policy 0, policy_version 50440 (0.0009) [2023-10-12 22:07:20,356][44958] Updated weights for policy 0, policy_version 50450 (0.0009) [2023-10-12 22:07:20,732][44958] Updated weights for policy 0, policy_version 50460 (0.0009) [2023-10-12 22:07:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 103612416. Throughput: 0: 1647.7, 1: 1646.1. Samples: 25904928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:21,443][43579] Avg episode reward: [(0, '275.050'), (1, '277.710')] [2023-10-12 22:07:23,390][44959] Updated weights for policy 1, policy_version 50730 (0.0007) [2023-10-12 22:07:23,755][44959] Updated weights for policy 1, policy_version 50740 (0.0010) [2023-10-12 22:07:24,136][44959] Updated weights for policy 1, policy_version 50750 (0.0010) [2023-10-12 22:07:24,695][44958] Updated weights for policy 0, policy_version 50470 (0.0007) [2023-10-12 22:07:25,081][44958] Updated weights for policy 0, policy_version 50480 (0.0008) [2023-10-12 22:07:25,451][44958] Updated weights for policy 0, policy_version 50490 (0.0008) [2023-10-12 22:07:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 103677952. Throughput: 0: 1640.3, 1: 1661.4. Samples: 25924580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:26,444][43579] Avg episode reward: [(0, '265.420'), (1, '274.040')] [2023-10-12 22:07:28,411][44959] Updated weights for policy 1, policy_version 50760 (0.0007) [2023-10-12 22:07:28,795][44959] Updated weights for policy 1, policy_version 50770 (0.0008) [2023-10-12 22:07:29,166][44959] Updated weights for policy 1, policy_version 50780 (0.0009) [2023-10-12 22:07:29,610][44958] Updated weights for policy 0, policy_version 50500 (0.0009) [2023-10-12 22:07:29,979][44958] Updated weights for policy 0, policy_version 50510 (0.0010) [2023-10-12 22:07:30,363][44958] Updated weights for policy 0, policy_version 50520 (0.0011) [2023-10-12 22:07:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 103743488. Throughput: 0: 1649.6, 1: 1660.0. Samples: 25944430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:31,443][43579] Avg episode reward: [(0, '259.430'), (1, '277.830')] [2023-10-12 22:07:33,254][44959] Updated weights for policy 1, policy_version 50790 (0.0010) [2023-10-12 22:07:33,611][44959] Updated weights for policy 1, policy_version 50800 (0.0009) [2023-10-12 22:07:33,978][44959] Updated weights for policy 1, policy_version 50810 (0.0008) [2023-10-12 22:07:34,359][44958] Updated weights for policy 0, policy_version 50530 (0.0008) [2023-10-12 22:07:34,727][44958] Updated weights for policy 0, policy_version 50540 (0.0008) [2023-10-12 22:07:35,111][44958] Updated weights for policy 0, policy_version 50550 (0.0008) [2023-10-12 22:07:35,479][44958] Updated weights for policy 0, policy_version 50560 (0.0010) [2023-10-12 22:07:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 103809024. Throughput: 0: 1653.8, 1: 1645.9. Samples: 25954750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:36,444][43579] Avg episode reward: [(0, '265.330'), (1, '281.330')] [2023-10-12 22:07:37,918][44959] Updated weights for policy 1, policy_version 50820 (0.0007) [2023-10-12 22:07:38,285][44959] Updated weights for policy 1, policy_version 50830 (0.0007) [2023-10-12 22:07:38,656][44959] Updated weights for policy 1, policy_version 50840 (0.0007) [2023-10-12 22:07:39,764][44958] Updated weights for policy 0, policy_version 50570 (0.0007) [2023-10-12 22:07:40,129][44958] Updated weights for policy 0, policy_version 50580 (0.0007) [2023-10-12 22:07:40,497][44958] Updated weights for policy 0, policy_version 50590 (0.0007) [2023-10-12 22:07:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 103874560. Throughput: 0: 1638.4, 1: 1661.4. Samples: 25974184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:41,444][43579] Avg episode reward: [(0, '268.080'), (1, '280.870')] [2023-10-12 22:07:42,773][44959] Updated weights for policy 1, policy_version 50850 (0.0008) [2023-10-12 22:07:43,142][44959] Updated weights for policy 1, policy_version 50860 (0.0008) [2023-10-12 22:07:43,515][44959] Updated weights for policy 1, policy_version 50870 (0.0008) [2023-10-12 22:07:43,876][44959] Updated weights for policy 1, policy_version 50880 (0.0009) [2023-10-12 22:07:44,720][44958] Updated weights for policy 0, policy_version 50600 (0.0007) [2023-10-12 22:07:45,092][44958] Updated weights for policy 0, policy_version 50610 (0.0009) [2023-10-12 22:07:45,466][44958] Updated weights for policy 0, policy_version 50620 (0.0008) [2023-10-12 22:07:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 103940096. Throughput: 0: 1646.7, 1: 1661.5. Samples: 25993834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:46,443][43579] Avg episode reward: [(0, '263.950'), (1, '281.540')] [2023-10-12 22:07:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000050624_51838976.pth... [2023-10-12 22:07:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000050880_52101120.pth... [2023-10-12 22:07:46,483][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000049344_50528256.pth [2023-10-12 22:07:46,491][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000049088_50266112.pth [2023-10-12 22:07:48,038][44959] Updated weights for policy 1, policy_version 50890 (0.0008) [2023-10-12 22:07:48,401][44959] Updated weights for policy 1, policy_version 50900 (0.0010) [2023-10-12 22:07:48,771][44959] Updated weights for policy 1, policy_version 50910 (0.0010) [2023-10-12 22:07:49,802][44958] Updated weights for policy 0, policy_version 50630 (0.0009) [2023-10-12 22:07:50,191][44958] Updated weights for policy 0, policy_version 50640 (0.0009) [2023-10-12 22:07:50,573][44958] Updated weights for policy 0, policy_version 50650 (0.0009) [2023-10-12 22:07:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104005632. Throughput: 0: 1649.0, 1: 1647.9. Samples: 26004026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:51,443][43579] Avg episode reward: [(0, '271.110'), (1, '277.460')] [2023-10-12 22:07:53,076][44959] Updated weights for policy 1, policy_version 50920 (0.0008) [2023-10-12 22:07:53,446][44959] Updated weights for policy 1, policy_version 50930 (0.0010) [2023-10-12 22:07:53,801][44959] Updated weights for policy 1, policy_version 50940 (0.0009) [2023-10-12 22:07:54,438][44958] Updated weights for policy 0, policy_version 50660 (0.0009) [2023-10-12 22:07:54,822][44958] Updated weights for policy 0, policy_version 50670 (0.0010) [2023-10-12 22:07:55,198][44958] Updated weights for policy 0, policy_version 50680 (0.0009) [2023-10-12 22:07:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104071168. Throughput: 0: 1634.8, 1: 1663.0. Samples: 26023304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:07:56,444][43579] Avg episode reward: [(0, '277.460'), (1, '277.300')] [2023-10-12 22:07:58,172][44959] Updated weights for policy 1, policy_version 50950 (0.0009) [2023-10-12 22:07:58,549][44959] Updated weights for policy 1, policy_version 50960 (0.0009) [2023-10-12 22:07:58,914][44959] Updated weights for policy 1, policy_version 50970 (0.0008) [2023-10-12 22:07:59,477][44958] Updated weights for policy 0, policy_version 50690 (0.0009) [2023-10-12 22:07:59,853][44958] Updated weights for policy 0, policy_version 50700 (0.0008) [2023-10-12 22:08:00,216][44958] Updated weights for policy 0, policy_version 50710 (0.0011) [2023-10-12 22:08:00,597][44958] Updated weights for policy 0, policy_version 50720 (0.0009) [2023-10-12 22:08:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 104136704. Throughput: 0: 1645.4, 1: 1658.3. Samples: 26043106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:08:01,444][43579] Avg episode reward: [(0, '283.840'), (1, '279.720')] [2023-10-12 22:08:02,875][44959] Updated weights for policy 1, policy_version 50980 (0.0008) [2023-10-12 22:08:03,239][44959] Updated weights for policy 1, policy_version 50990 (0.0008) [2023-10-12 22:08:03,607][44959] Updated weights for policy 1, policy_version 51000 (0.0010) [2023-10-12 22:08:04,753][44958] Updated weights for policy 0, policy_version 50730 (0.0010) [2023-10-12 22:08:05,133][44958] Updated weights for policy 0, policy_version 50740 (0.0009) [2023-10-12 22:08:05,507][44958] Updated weights for policy 0, policy_version 50750 (0.0010) [2023-10-12 22:08:06,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 104202240. Throughput: 0: 1645.9, 1: 1650.1. Samples: 26053246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:08:06,443][43579] Avg episode reward: [(0, '280.890'), (1, '277.710')] [2023-10-12 22:08:07,859][44959] Updated weights for policy 1, policy_version 51010 (0.0010) [2023-10-12 22:08:08,231][44959] Updated weights for policy 1, policy_version 51020 (0.0007) [2023-10-12 22:08:08,602][44959] Updated weights for policy 1, policy_version 51030 (0.0009) [2023-10-12 22:08:08,969][44959] Updated weights for policy 1, policy_version 51040 (0.0008) [2023-10-12 22:08:09,747][44958] Updated weights for policy 0, policy_version 50760 (0.0008) [2023-10-12 22:08:10,115][44958] Updated weights for policy 0, policy_version 50770 (0.0008) [2023-10-12 22:08:10,484][44958] Updated weights for policy 0, policy_version 50780 (0.0009) [2023-10-12 22:08:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 104267776. Throughput: 0: 1640.8, 1: 1651.7. Samples: 26072740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:08:11,443][43579] Avg episode reward: [(0, '284.400'), (1, '277.860')] [2023-10-12 22:08:13,134][44959] Updated weights for policy 1, policy_version 51050 (0.0010) [2023-10-12 22:08:13,505][44959] Updated weights for policy 1, policy_version 51060 (0.0011) [2023-10-12 22:08:13,877][44959] Updated weights for policy 1, policy_version 51070 (0.0011) [2023-10-12 22:08:14,463][44958] Updated weights for policy 0, policy_version 50790 (0.0009) [2023-10-12 22:08:14,834][44958] Updated weights for policy 0, policy_version 50800 (0.0008) [2023-10-12 22:08:15,211][44958] Updated weights for policy 0, policy_version 50810 (0.0010) [2023-10-12 22:08:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 104333312. Throughput: 0: 1641.1, 1: 1648.9. Samples: 26092482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:08:16,443][43579] Avg episode reward: [(0, '282.750'), (1, '273.990')] [2023-10-12 22:08:18,226][44959] Updated weights for policy 1, policy_version 51080 (0.0010) [2023-10-12 22:08:18,593][44959] Updated weights for policy 1, policy_version 51090 (0.0009) [2023-10-12 22:08:18,962][44959] Updated weights for policy 1, policy_version 51100 (0.0009) [2023-10-12 22:08:19,387][44958] Updated weights for policy 0, policy_version 50820 (0.0008) [2023-10-12 22:08:19,762][44958] Updated weights for policy 0, policy_version 50830 (0.0008) [2023-10-12 22:08:20,130][44958] Updated weights for policy 0, policy_version 50840 (0.0011) [2023-10-12 22:08:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104398848. Throughput: 0: 1641.9, 1: 1647.8. Samples: 26102784. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-12 22:08:21,443][43579] Avg episode reward: [(0, '280.180'), (1, '274.070')] [2023-10-12 22:08:23,116][44959] Updated weights for policy 1, policy_version 51110 (0.0010) [2023-10-12 22:08:23,490][44959] Updated weights for policy 1, policy_version 51120 (0.0010) [2023-10-12 22:08:23,869][44959] Updated weights for policy 1, policy_version 51130 (0.0009) [2023-10-12 22:08:24,364][44958] Updated weights for policy 0, policy_version 50850 (0.0009) [2023-10-12 22:08:24,739][44958] Updated weights for policy 0, policy_version 50860 (0.0008) [2023-10-12 22:08:25,095][44958] Updated weights for policy 0, policy_version 50870 (0.0009) [2023-10-12 22:08:25,476][44958] Updated weights for policy 0, policy_version 50880 (0.0010) [2023-10-12 22:08:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104464384. Throughput: 0: 1641.4, 1: 1640.7. Samples: 26121880. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-12 22:08:26,444][43579] Avg episode reward: [(0, '279.160'), (1, '275.880')] [2023-10-12 22:08:27,976][44959] Updated weights for policy 1, policy_version 51140 (0.0007) [2023-10-12 22:08:28,342][44959] Updated weights for policy 1, policy_version 51150 (0.0008) [2023-10-12 22:08:28,713][44959] Updated weights for policy 1, policy_version 51160 (0.0007) [2023-10-12 22:08:29,770][44958] Updated weights for policy 0, policy_version 50890 (0.0009) [2023-10-12 22:08:30,146][44958] Updated weights for policy 0, policy_version 50900 (0.0008) [2023-10-12 22:08:30,513][44958] Updated weights for policy 0, policy_version 50910 (0.0008) [2023-10-12 22:08:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104529920. Throughput: 0: 1640.5, 1: 1640.8. Samples: 26141496. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-12 22:08:31,443][43579] Avg episode reward: [(0, '275.880'), (1, '274.400')] [2023-10-12 22:08:32,921][44959] Updated weights for policy 1, policy_version 51170 (0.0009) [2023-10-12 22:08:33,287][44959] Updated weights for policy 1, policy_version 51180 (0.0009) [2023-10-12 22:08:33,661][44959] Updated weights for policy 1, policy_version 51190 (0.0009) [2023-10-12 22:08:34,027][44959] Updated weights for policy 1, policy_version 51200 (0.0007) [2023-10-12 22:08:34,672][44958] Updated weights for policy 0, policy_version 50920 (0.0008) [2023-10-12 22:08:35,055][44958] Updated weights for policy 0, policy_version 50930 (0.0009) [2023-10-12 22:08:35,425][44958] Updated weights for policy 0, policy_version 50940 (0.0007) [2023-10-12 22:08:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104595456. Throughput: 0: 1643.5, 1: 1638.9. Samples: 26151732. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-12 22:08:36,444][43579] Avg episode reward: [(0, '277.240'), (1, '277.200')] [2023-10-12 22:08:37,961][44959] Updated weights for policy 1, policy_version 51210 (0.0007) [2023-10-12 22:08:38,333][44959] Updated weights for policy 1, policy_version 51220 (0.0009) [2023-10-12 22:08:38,711][44959] Updated weights for policy 1, policy_version 51230 (0.0008) [2023-10-12 22:08:39,573][44958] Updated weights for policy 0, policy_version 50950 (0.0008) [2023-10-12 22:08:39,945][44958] Updated weights for policy 0, policy_version 50960 (0.0007) [2023-10-12 22:08:40,312][44958] Updated weights for policy 0, policy_version 50970 (0.0007) [2023-10-12 22:08:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104660992. Throughput: 0: 1641.8, 1: 1643.6. Samples: 26171150. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-12 22:08:41,444][43579] Avg episode reward: [(0, '277.430'), (1, '274.450')] [2023-10-12 22:08:42,943][44959] Updated weights for policy 1, policy_version 51240 (0.0010) [2023-10-12 22:08:43,320][44959] Updated weights for policy 1, policy_version 51250 (0.0008) [2023-10-12 22:08:43,689][44959] Updated weights for policy 1, policy_version 51260 (0.0009) [2023-10-12 22:08:44,422][44958] Updated weights for policy 0, policy_version 50980 (0.0007) [2023-10-12 22:08:44,795][44958] Updated weights for policy 0, policy_version 50990 (0.0007) [2023-10-12 22:08:45,165][44958] Updated weights for policy 0, policy_version 51000 (0.0009) [2023-10-12 22:08:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104726528. Throughput: 0: 1640.4, 1: 1646.9. Samples: 26191032. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-12 22:08:46,443][43579] Avg episode reward: [(0, '282.010'), (1, '268.050')] [2023-10-12 22:08:47,876][44959] Updated weights for policy 1, policy_version 51270 (0.0009) [2023-10-12 22:08:48,244][44959] Updated weights for policy 1, policy_version 51280 (0.0009) [2023-10-12 22:08:48,618][44959] Updated weights for policy 1, policy_version 51290 (0.0008) [2023-10-12 22:08:49,616][44958] Updated weights for policy 0, policy_version 51010 (0.0008) [2023-10-12 22:08:49,988][44958] Updated weights for policy 0, policy_version 51020 (0.0010) [2023-10-12 22:08:50,364][44958] Updated weights for policy 0, policy_version 51030 (0.0010) [2023-10-12 22:08:50,737][44958] Updated weights for policy 0, policy_version 51040 (0.0010) [2023-10-12 22:08:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104792064. Throughput: 0: 1636.8, 1: 1641.1. Samples: 26200748. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-12 22:08:51,443][43579] Avg episode reward: [(0, '285.090'), (1, '264.940')] [2023-10-12 22:08:52,608][44959] Updated weights for policy 1, policy_version 51300 (0.0009) [2023-10-12 22:08:52,968][44959] Updated weights for policy 1, policy_version 51310 (0.0009) [2023-10-12 22:08:53,330][44959] Updated weights for policy 1, policy_version 51320 (0.0009) [2023-10-12 22:08:54,972][44958] Updated weights for policy 0, policy_version 51050 (0.0007) [2023-10-12 22:08:55,359][44958] Updated weights for policy 0, policy_version 51060 (0.0008) [2023-10-12 22:08:55,730][44958] Updated weights for policy 0, policy_version 51070 (0.0010) [2023-10-12 22:08:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104857600. Throughput: 0: 1637.0, 1: 1645.8. Samples: 26220466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:08:56,443][43579] Avg episode reward: [(0, '287.450'), (1, '263.300')] [2023-10-12 22:08:56,444][44518] Saving new best policy, reward=287.450! [2023-10-12 22:08:57,446][44959] Updated weights for policy 1, policy_version 51330 (0.0011) [2023-10-12 22:08:57,817][44959] Updated weights for policy 1, policy_version 51340 (0.0010) [2023-10-12 22:08:58,190][44959] Updated weights for policy 1, policy_version 51350 (0.0009) [2023-10-12 22:08:58,558][44959] Updated weights for policy 1, policy_version 51360 (0.0011) [2023-10-12 22:08:59,984][44958] Updated weights for policy 0, policy_version 51080 (0.0010) [2023-10-12 22:09:00,357][44958] Updated weights for policy 0, policy_version 51090 (0.0008) [2023-10-12 22:09:00,725][44958] Updated weights for policy 0, policy_version 51100 (0.0009) [2023-10-12 22:09:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 104923136. Throughput: 0: 1631.2, 1: 1644.7. Samples: 26239896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:09:01,443][43579] Avg episode reward: [(0, '287.960'), (1, '262.130')] [2023-10-12 22:09:01,454][44518] Saving new best policy, reward=287.960! [2023-10-12 22:09:02,702][44959] Updated weights for policy 1, policy_version 51370 (0.0007) [2023-10-12 22:09:03,063][44959] Updated weights for policy 1, policy_version 51380 (0.0007) [2023-10-12 22:09:03,435][44959] Updated weights for policy 1, policy_version 51390 (0.0008) [2023-10-12 22:09:04,920][44958] Updated weights for policy 0, policy_version 51110 (0.0009) [2023-10-12 22:09:05,296][44958] Updated weights for policy 0, policy_version 51120 (0.0008) [2023-10-12 22:09:05,664][44958] Updated weights for policy 0, policy_version 51130 (0.0008) [2023-10-12 22:09:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 104988672. Throughput: 0: 1628.2, 1: 1640.6. Samples: 26249882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:09:06,444][43579] Avg episode reward: [(0, '288.160'), (1, '264.580')] [2023-10-12 22:09:06,445][44518] Saving new best policy, reward=288.160! [2023-10-12 22:09:07,776][44959] Updated weights for policy 1, policy_version 51400 (0.0007) [2023-10-12 22:09:08,156][44959] Updated weights for policy 1, policy_version 51410 (0.0008) [2023-10-12 22:09:08,524][44959] Updated weights for policy 1, policy_version 51420 (0.0009) [2023-10-12 22:09:09,609][44958] Updated weights for policy 0, policy_version 51140 (0.0007) [2023-10-12 22:09:09,990][44958] Updated weights for policy 0, policy_version 51150 (0.0007) [2023-10-12 22:09:10,361][44958] Updated weights for policy 0, policy_version 51160 (0.0007) [2023-10-12 22:09:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105054208. Throughput: 0: 1636.5, 1: 1646.8. Samples: 26269628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:09:11,444][43579] Avg episode reward: [(0, '285.510'), (1, '264.050')] [2023-10-12 22:09:12,574][44959] Updated weights for policy 1, policy_version 51430 (0.0008) [2023-10-12 22:09:12,940][44959] Updated weights for policy 1, policy_version 51440 (0.0008) [2023-10-12 22:09:13,307][44959] Updated weights for policy 1, policy_version 51450 (0.0007) [2023-10-12 22:09:14,630][44958] Updated weights for policy 0, policy_version 51170 (0.0007) [2023-10-12 22:09:14,997][44958] Updated weights for policy 0, policy_version 51180 (0.0008) [2023-10-12 22:09:15,361][44958] Updated weights for policy 0, policy_version 51190 (0.0008) [2023-10-12 22:09:15,735][44958] Updated weights for policy 0, policy_version 51200 (0.0008) [2023-10-12 22:09:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105119744. Throughput: 0: 1635.0, 1: 1651.2. Samples: 26289372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:09:16,443][43579] Avg episode reward: [(0, '287.010'), (1, '277.310')] [2023-10-12 22:09:17,598][44959] Updated weights for policy 1, policy_version 51460 (0.0010) [2023-10-12 22:09:17,962][44959] Updated weights for policy 1, policy_version 51470 (0.0007) [2023-10-12 22:09:18,334][44959] Updated weights for policy 1, policy_version 51480 (0.0008) [2023-10-12 22:09:19,972][44958] Updated weights for policy 0, policy_version 51210 (0.0011) [2023-10-12 22:09:20,349][44958] Updated weights for policy 0, policy_version 51220 (0.0009) [2023-10-12 22:09:20,719][44958] Updated weights for policy 0, policy_version 51230 (0.0009) [2023-10-12 22:09:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105185280. Throughput: 0: 1632.4, 1: 1646.6. Samples: 26299290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:09:21,444][43579] Avg episode reward: [(0, '283.170'), (1, '279.800')] [2023-10-12 22:09:22,441][44959] Updated weights for policy 1, policy_version 51490 (0.0009) [2023-10-12 22:09:22,806][44959] Updated weights for policy 1, policy_version 51500 (0.0011) [2023-10-12 22:09:23,171][44959] Updated weights for policy 1, policy_version 51510 (0.0007) [2023-10-12 22:09:23,535][44959] Updated weights for policy 1, policy_version 51520 (0.0008) [2023-10-12 22:09:24,891][44958] Updated weights for policy 0, policy_version 51240 (0.0008) [2023-10-12 22:09:25,259][44958] Updated weights for policy 0, policy_version 51250 (0.0008) [2023-10-12 22:09:25,637][44958] Updated weights for policy 0, policy_version 51260 (0.0009) [2023-10-12 22:09:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105250816. Throughput: 0: 1636.2, 1: 1648.2. Samples: 26318946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:09:26,443][43579] Avg episode reward: [(0, '279.470'), (1, '282.790')] [2023-10-12 22:09:27,688][44959] Updated weights for policy 1, policy_version 51530 (0.0008) [2023-10-12 22:09:28,063][44959] Updated weights for policy 1, policy_version 51540 (0.0008) [2023-10-12 22:09:28,435][44959] Updated weights for policy 1, policy_version 51550 (0.0007) [2023-10-12 22:09:29,930][44958] Updated weights for policy 0, policy_version 51270 (0.0009) [2023-10-12 22:09:30,296][44958] Updated weights for policy 0, policy_version 51280 (0.0011) [2023-10-12 22:09:30,664][44958] Updated weights for policy 0, policy_version 51290 (0.0011) [2023-10-12 22:09:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105316352. Throughput: 0: 1628.2, 1: 1644.8. Samples: 26338320. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-12 22:09:31,443][43579] Avg episode reward: [(0, '281.520'), (1, '286.290')] [2023-10-12 22:09:32,705][44959] Updated weights for policy 1, policy_version 51560 (0.0008) [2023-10-12 22:09:33,071][44959] Updated weights for policy 1, policy_version 51570 (0.0008) [2023-10-12 22:09:33,447][44959] Updated weights for policy 1, policy_version 51580 (0.0009) [2023-10-12 22:09:34,735][44958] Updated weights for policy 0, policy_version 51300 (0.0007) [2023-10-12 22:09:35,112][44958] Updated weights for policy 0, policy_version 51310 (0.0008) [2023-10-12 22:09:35,486][44958] Updated weights for policy 0, policy_version 51320 (0.0009) [2023-10-12 22:09:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105381888. Throughput: 0: 1635.6, 1: 1649.2. Samples: 26348564. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-12 22:09:36,443][43579] Avg episode reward: [(0, '274.900'), (1, '284.390')] [2023-10-12 22:09:37,589][44959] Updated weights for policy 1, policy_version 51590 (0.0010) [2023-10-12 22:09:37,959][44959] Updated weights for policy 1, policy_version 51600 (0.0009) [2023-10-12 22:09:38,332][44959] Updated weights for policy 1, policy_version 51610 (0.0009) [2023-10-12 22:09:39,535][44958] Updated weights for policy 0, policy_version 51330 (0.0007) [2023-10-12 22:09:39,914][44958] Updated weights for policy 0, policy_version 51340 (0.0008) [2023-10-12 22:09:40,280][44958] Updated weights for policy 0, policy_version 51350 (0.0008) [2023-10-12 22:09:40,661][44958] Updated weights for policy 0, policy_version 51360 (0.0008) [2023-10-12 22:09:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105447424. Throughput: 0: 1635.7, 1: 1646.0. Samples: 26368146. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-12 22:09:41,443][43579] Avg episode reward: [(0, '274.090'), (1, '285.740')] [2023-10-12 22:09:42,541][44959] Updated weights for policy 1, policy_version 51620 (0.0008) [2023-10-12 22:09:42,908][44959] Updated weights for policy 1, policy_version 51630 (0.0007) [2023-10-12 22:09:43,294][44959] Updated weights for policy 1, policy_version 51640 (0.0008) [2023-10-12 22:09:44,957][44958] Updated weights for policy 0, policy_version 51370 (0.0008) [2023-10-12 22:09:45,337][44958] Updated weights for policy 0, policy_version 51380 (0.0009) [2023-10-12 22:09:45,715][44958] Updated weights for policy 0, policy_version 51390 (0.0008) [2023-10-12 22:09:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 105512960. Throughput: 0: 1636.8, 1: 1651.4. Samples: 26387866. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-12 22:09:46,444][43579] Avg episode reward: [(0, '271.550'), (1, '284.080')] [2023-10-12 22:09:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000051648_52887552.pth... [2023-10-12 22:09:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000051392_52625408.pth... [2023-10-12 22:09:46,495][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000049856_51052544.pth [2023-10-12 22:09:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000050112_51314688.pth [2023-10-12 22:09:47,466][44959] Updated weights for policy 1, policy_version 51650 (0.0009) [2023-10-12 22:09:47,842][44959] Updated weights for policy 1, policy_version 51660 (0.0007) [2023-10-12 22:09:48,214][44959] Updated weights for policy 1, policy_version 51670 (0.0008) [2023-10-12 22:09:48,575][44959] Updated weights for policy 1, policy_version 51680 (0.0009) [2023-10-12 22:09:49,838][44958] Updated weights for policy 0, policy_version 51400 (0.0009) [2023-10-12 22:09:50,202][44958] Updated weights for policy 0, policy_version 51410 (0.0008) [2023-10-12 22:09:50,576][44958] Updated weights for policy 0, policy_version 51420 (0.0008) [2023-10-12 22:09:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105578496. Throughput: 0: 1636.3, 1: 1651.8. Samples: 26397846. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-12 22:09:51,443][43579] Avg episode reward: [(0, '267.570'), (1, '280.740')] [2023-10-12 22:09:52,819][44959] Updated weights for policy 1, policy_version 51690 (0.0009) [2023-10-12 22:09:53,192][44959] Updated weights for policy 1, policy_version 51700 (0.0008) [2023-10-12 22:09:53,558][44959] Updated weights for policy 1, policy_version 51710 (0.0008) [2023-10-12 22:09:54,765][44958] Updated weights for policy 0, policy_version 51430 (0.0008) [2023-10-12 22:09:55,135][44958] Updated weights for policy 0, policy_version 51440 (0.0007) [2023-10-12 22:09:55,507][44958] Updated weights for policy 0, policy_version 51450 (0.0007) [2023-10-12 22:09:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105644032. Throughput: 0: 1639.2, 1: 1648.9. Samples: 26417592. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-12 22:09:56,444][43579] Avg episode reward: [(0, '263.450'), (1, '281.200')] [2023-10-12 22:09:57,883][44959] Updated weights for policy 1, policy_version 51720 (0.0010) [2023-10-12 22:09:58,269][44959] Updated weights for policy 1, policy_version 51730 (0.0009) [2023-10-12 22:09:58,636][44959] Updated weights for policy 1, policy_version 51740 (0.0008) [2023-10-12 22:09:59,692][44958] Updated weights for policy 0, policy_version 51460 (0.0009) [2023-10-12 22:10:00,061][44958] Updated weights for policy 0, policy_version 51470 (0.0011) [2023-10-12 22:10:00,430][44958] Updated weights for policy 0, policy_version 51480 (0.0009) [2023-10-12 22:10:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105709568. Throughput: 0: 1637.2, 1: 1639.9. Samples: 26436838. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-12 22:10:01,443][43579] Avg episode reward: [(0, '260.960'), (1, '278.270')] [2023-10-12 22:10:02,537][44959] Updated weights for policy 1, policy_version 51750 (0.0009) [2023-10-12 22:10:02,895][44959] Updated weights for policy 1, policy_version 51760 (0.0009) [2023-10-12 22:10:03,266][44959] Updated weights for policy 1, policy_version 51770 (0.0008) [2023-10-12 22:10:04,631][44958] Updated weights for policy 0, policy_version 51490 (0.0009) [2023-10-12 22:10:05,048][44958] Updated weights for policy 0, policy_version 51500 (0.0009) [2023-10-12 22:10:05,437][44958] Updated weights for policy 0, policy_version 51510 (0.0009) [2023-10-12 22:10:05,802][44958] Updated weights for policy 0, policy_version 51520 (0.0008) [2023-10-12 22:10:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105775104. Throughput: 0: 1640.5, 1: 1646.6. Samples: 26447208. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 22:10:06,444][43579] Avg episode reward: [(0, '267.060'), (1, '279.670')] [2023-10-12 22:10:07,173][44959] Updated weights for policy 1, policy_version 51780 (0.0008) [2023-10-12 22:10:07,541][44959] Updated weights for policy 1, policy_version 51790 (0.0007) [2023-10-12 22:10:07,905][44959] Updated weights for policy 1, policy_version 51800 (0.0008) [2023-10-12 22:10:09,983][44958] Updated weights for policy 0, policy_version 51530 (0.0008) [2023-10-12 22:10:10,355][44958] Updated weights for policy 0, policy_version 51540 (0.0008) [2023-10-12 22:10:10,727][44958] Updated weights for policy 0, policy_version 51550 (0.0008) [2023-10-12 22:10:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105840640. Throughput: 0: 1644.8, 1: 1641.3. Samples: 26466824. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 22:10:11,443][43579] Avg episode reward: [(0, '265.710'), (1, '275.320')] [2023-10-12 22:10:12,379][44959] Updated weights for policy 1, policy_version 51810 (0.0009) [2023-10-12 22:10:12,749][44959] Updated weights for policy 1, policy_version 51820 (0.0009) [2023-10-12 22:10:13,116][44959] Updated weights for policy 1, policy_version 51830 (0.0009) [2023-10-12 22:10:13,477][44959] Updated weights for policy 1, policy_version 51840 (0.0007) [2023-10-12 22:10:14,759][44958] Updated weights for policy 0, policy_version 51560 (0.0008) [2023-10-12 22:10:15,131][44958] Updated weights for policy 0, policy_version 51570 (0.0009) [2023-10-12 22:10:15,513][44958] Updated weights for policy 0, policy_version 51580 (0.0009) [2023-10-12 22:10:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105906176. Throughput: 0: 1644.7, 1: 1643.8. Samples: 26486302. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 22:10:16,444][43579] Avg episode reward: [(0, '273.020'), (1, '275.760')] [2023-10-12 22:10:17,444][44959] Updated weights for policy 1, policy_version 51850 (0.0010) [2023-10-12 22:10:17,813][44959] Updated weights for policy 1, policy_version 51860 (0.0010) [2023-10-12 22:10:18,181][44959] Updated weights for policy 1, policy_version 51870 (0.0008) [2023-10-12 22:10:19,722][44958] Updated weights for policy 0, policy_version 51590 (0.0008) [2023-10-12 22:10:20,090][44958] Updated weights for policy 0, policy_version 51600 (0.0010) [2023-10-12 22:10:20,469][44958] Updated weights for policy 0, policy_version 51610 (0.0011) [2023-10-12 22:10:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 105971712. Throughput: 0: 1640.1, 1: 1645.8. Samples: 26496430. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 22:10:21,443][43579] Avg episode reward: [(0, '275.580'), (1, '275.680')] [2023-10-12 22:10:22,445][44959] Updated weights for policy 1, policy_version 51880 (0.0009) [2023-10-12 22:10:22,815][44959] Updated weights for policy 1, policy_version 51890 (0.0010) [2023-10-12 22:10:23,184][44959] Updated weights for policy 1, policy_version 51900 (0.0010) [2023-10-12 22:10:24,762][44958] Updated weights for policy 0, policy_version 51620 (0.0009) [2023-10-12 22:10:25,137][44958] Updated weights for policy 0, policy_version 51630 (0.0008) [2023-10-12 22:10:25,516][44958] Updated weights for policy 0, policy_version 51640 (0.0008) [2023-10-12 22:10:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106037248. Throughput: 0: 1639.9, 1: 1647.2. Samples: 26516062. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 22:10:26,444][43579] Avg episode reward: [(0, '275.640'), (1, '276.650')] [2023-10-12 22:10:27,216][44959] Updated weights for policy 1, policy_version 51910 (0.0008) [2023-10-12 22:10:27,595][44959] Updated weights for policy 1, policy_version 51920 (0.0008) [2023-10-12 22:10:27,964][44959] Updated weights for policy 1, policy_version 51930 (0.0007) [2023-10-12 22:10:29,654][44958] Updated weights for policy 0, policy_version 51650 (0.0009) [2023-10-12 22:10:30,027][44958] Updated weights for policy 0, policy_version 51660 (0.0008) [2023-10-12 22:10:30,395][44958] Updated weights for policy 0, policy_version 51670 (0.0010) [2023-10-12 22:10:30,762][44958] Updated weights for policy 0, policy_version 51680 (0.0008) [2023-10-12 22:10:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106102784. Throughput: 0: 1640.1, 1: 1645.2. Samples: 26535700. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 22:10:31,443][43579] Avg episode reward: [(0, '277.580'), (1, '275.530')] [2023-10-12 22:10:32,010][44959] Updated weights for policy 1, policy_version 51940 (0.0009) [2023-10-12 22:10:32,382][44959] Updated weights for policy 1, policy_version 51950 (0.0008) [2023-10-12 22:10:32,747][44959] Updated weights for policy 1, policy_version 51960 (0.0008) [2023-10-12 22:10:34,778][44958] Updated weights for policy 0, policy_version 51690 (0.0009) [2023-10-12 22:10:35,152][44958] Updated weights for policy 0, policy_version 51700 (0.0007) [2023-10-12 22:10:35,523][44958] Updated weights for policy 0, policy_version 51710 (0.0007) [2023-10-12 22:10:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106168320. Throughput: 0: 1641.1, 1: 1649.6. Samples: 26545928. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-12 22:10:36,444][43579] Avg episode reward: [(0, '279.430'), (1, '277.020')] [2023-10-12 22:10:36,998][44959] Updated weights for policy 1, policy_version 51970 (0.0009) [2023-10-12 22:10:37,361][44959] Updated weights for policy 1, policy_version 51980 (0.0011) [2023-10-12 22:10:37,730][44959] Updated weights for policy 1, policy_version 51990 (0.0011) [2023-10-12 22:10:38,102][44959] Updated weights for policy 1, policy_version 52000 (0.0009) [2023-10-12 22:10:39,677][44958] Updated weights for policy 0, policy_version 51720 (0.0008) [2023-10-12 22:10:40,060][44958] Updated weights for policy 0, policy_version 51730 (0.0009) [2023-10-12 22:10:40,437][44958] Updated weights for policy 0, policy_version 51740 (0.0008) [2023-10-12 22:10:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106233856. Throughput: 0: 1638.2, 1: 1647.9. Samples: 26565466. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 22:10:41,443][43579] Avg episode reward: [(0, '275.090'), (1, '280.220')] [2023-10-12 22:10:42,443][44959] Updated weights for policy 1, policy_version 52010 (0.0010) [2023-10-12 22:10:42,819][44959] Updated weights for policy 1, policy_version 52020 (0.0008) [2023-10-12 22:10:43,187][44959] Updated weights for policy 1, policy_version 52030 (0.0007) [2023-10-12 22:10:44,722][44958] Updated weights for policy 0, policy_version 51750 (0.0008) [2023-10-12 22:10:45,098][44958] Updated weights for policy 0, policy_version 51760 (0.0007) [2023-10-12 22:10:45,474][44958] Updated weights for policy 0, policy_version 51770 (0.0010) [2023-10-12 22:10:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106299392. Throughput: 0: 1644.3, 1: 1657.1. Samples: 26585400. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 22:10:46,444][43579] Avg episode reward: [(0, '272.700'), (1, '282.640')] [2023-10-12 22:10:47,236][44959] Updated weights for policy 1, policy_version 52040 (0.0008) [2023-10-12 22:10:47,617][44959] Updated weights for policy 1, policy_version 52050 (0.0007) [2023-10-12 22:10:47,980][44959] Updated weights for policy 1, policy_version 52060 (0.0011) [2023-10-12 22:10:49,661][44958] Updated weights for policy 0, policy_version 51780 (0.0009) [2023-10-12 22:10:50,048][44958] Updated weights for policy 0, policy_version 51790 (0.0007) [2023-10-12 22:10:50,420][44958] Updated weights for policy 0, policy_version 51800 (0.0010) [2023-10-12 22:10:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106364928. Throughput: 0: 1642.7, 1: 1653.7. Samples: 26595546. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 22:10:51,444][43579] Avg episode reward: [(0, '270.450'), (1, '282.500')] [2023-10-12 22:10:52,147][44959] Updated weights for policy 1, policy_version 52070 (0.0008) [2023-10-12 22:10:52,516][44959] Updated weights for policy 1, policy_version 52080 (0.0009) [2023-10-12 22:10:52,875][44959] Updated weights for policy 1, policy_version 52090 (0.0008) [2023-10-12 22:10:54,538][44958] Updated weights for policy 0, policy_version 51810 (0.0008) [2023-10-12 22:10:54,910][44958] Updated weights for policy 0, policy_version 51820 (0.0011) [2023-10-12 22:10:55,281][44958] Updated weights for policy 0, policy_version 51830 (0.0008) [2023-10-12 22:10:55,660][44958] Updated weights for policy 0, policy_version 51840 (0.0007) [2023-10-12 22:10:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106430464. Throughput: 0: 1638.4, 1: 1659.0. Samples: 26615210. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 22:10:56,444][43579] Avg episode reward: [(0, '275.420'), (1, '282.240')] [2023-10-12 22:10:56,845][44959] Updated weights for policy 1, policy_version 52100 (0.0007) [2023-10-12 22:10:57,222][44959] Updated weights for policy 1, policy_version 52110 (0.0007) [2023-10-12 22:10:57,593][44959] Updated weights for policy 1, policy_version 52120 (0.0008) [2023-10-12 22:10:59,900][44958] Updated weights for policy 0, policy_version 51850 (0.0007) [2023-10-12 22:11:00,275][44958] Updated weights for policy 0, policy_version 51860 (0.0007) [2023-10-12 22:11:00,646][44958] Updated weights for policy 0, policy_version 51870 (0.0009) [2023-10-12 22:11:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 106496000. Throughput: 0: 1642.4, 1: 1658.8. Samples: 26634858. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 22:11:01,444][43579] Avg episode reward: [(0, '273.860'), (1, '282.700')] [2023-10-12 22:11:01,837][44959] Updated weights for policy 1, policy_version 52130 (0.0009) [2023-10-12 22:11:02,197][44959] Updated weights for policy 1, policy_version 52140 (0.0008) [2023-10-12 22:11:02,558][44959] Updated weights for policy 1, policy_version 52150 (0.0010) [2023-10-12 22:11:02,921][44959] Updated weights for policy 1, policy_version 52160 (0.0011) [2023-10-12 22:11:04,782][44958] Updated weights for policy 0, policy_version 51880 (0.0008) [2023-10-12 22:11:05,151][44958] Updated weights for policy 0, policy_version 51890 (0.0007) [2023-10-12 22:11:05,531][44958] Updated weights for policy 0, policy_version 51900 (0.0007) [2023-10-12 22:11:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106561536. Throughput: 0: 1645.8, 1: 1653.6. Samples: 26644900. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 22:11:06,443][43579] Avg episode reward: [(0, '273.930'), (1, '278.550')] [2023-10-12 22:11:07,235][44959] Updated weights for policy 1, policy_version 52170 (0.0010) [2023-10-12 22:11:07,597][44959] Updated weights for policy 1, policy_version 52180 (0.0009) [2023-10-12 22:11:07,961][44959] Updated weights for policy 1, policy_version 52190 (0.0007) [2023-10-12 22:11:09,708][44958] Updated weights for policy 0, policy_version 51910 (0.0008) [2023-10-12 22:11:10,080][44958] Updated weights for policy 0, policy_version 51920 (0.0007) [2023-10-12 22:11:10,457][44958] Updated weights for policy 0, policy_version 51930 (0.0008) [2023-10-12 22:11:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106627072. Throughput: 0: 1642.9, 1: 1654.9. Samples: 26664464. Policy #0 lag: (min: 1.0, avg: 4.8, max: 33.0) [2023-10-12 22:11:11,444][43579] Avg episode reward: [(0, '277.590'), (1, '272.480')] [2023-10-12 22:11:12,067][44959] Updated weights for policy 1, policy_version 52200 (0.0010) [2023-10-12 22:11:12,441][44959] Updated weights for policy 1, policy_version 52210 (0.0010) [2023-10-12 22:11:12,813][44959] Updated weights for policy 1, policy_version 52220 (0.0009) [2023-10-12 22:11:14,620][44958] Updated weights for policy 0, policy_version 51940 (0.0010) [2023-10-12 22:11:14,988][44958] Updated weights for policy 0, policy_version 51950 (0.0009) [2023-10-12 22:11:15,365][44958] Updated weights for policy 0, policy_version 51960 (0.0011) [2023-10-12 22:11:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106692608. Throughput: 0: 1647.9, 1: 1653.2. Samples: 26684246. Policy #0 lag: (min: 8.0, avg: 36.6, max: 40.0) [2023-10-12 22:11:16,443][43579] Avg episode reward: [(0, '278.860'), (1, '273.550')] [2023-10-12 22:11:16,973][44959] Updated weights for policy 1, policy_version 52230 (0.0009) [2023-10-12 22:11:17,344][44959] Updated weights for policy 1, policy_version 52240 (0.0007) [2023-10-12 22:11:17,713][44959] Updated weights for policy 1, policy_version 52250 (0.0009) [2023-10-12 22:11:19,437][44958] Updated weights for policy 0, policy_version 51970 (0.0010) [2023-10-12 22:11:19,808][44958] Updated weights for policy 0, policy_version 51980 (0.0008) [2023-10-12 22:11:20,178][44958] Updated weights for policy 0, policy_version 51990 (0.0008) [2023-10-12 22:11:20,549][44958] Updated weights for policy 0, policy_version 52000 (0.0010) [2023-10-12 22:11:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106758144. Throughput: 0: 1653.5, 1: 1647.0. Samples: 26694450. Policy #0 lag: (min: 8.0, avg: 36.6, max: 40.0) [2023-10-12 22:11:21,444][43579] Avg episode reward: [(0, '278.030'), (1, '277.050')] [2023-10-12 22:11:21,804][44959] Updated weights for policy 1, policy_version 52260 (0.0007) [2023-10-12 22:11:22,167][44959] Updated weights for policy 1, policy_version 52270 (0.0008) [2023-10-12 22:11:22,542][44959] Updated weights for policy 1, policy_version 52280 (0.0008) [2023-10-12 22:11:24,722][44958] Updated weights for policy 0, policy_version 52010 (0.0009) [2023-10-12 22:11:25,095][44958] Updated weights for policy 0, policy_version 52020 (0.0007) [2023-10-12 22:11:25,471][44958] Updated weights for policy 0, policy_version 52030 (0.0010) [2023-10-12 22:11:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106823680. Throughput: 0: 1651.5, 1: 1654.1. Samples: 26714216. Policy #0 lag: (min: 8.0, avg: 36.6, max: 40.0) [2023-10-12 22:11:26,444][43579] Avg episode reward: [(0, '276.670'), (1, '276.180')] [2023-10-12 22:11:26,571][44959] Updated weights for policy 1, policy_version 52290 (0.0008) [2023-10-12 22:11:26,934][44959] Updated weights for policy 1, policy_version 52300 (0.0007) [2023-10-12 22:11:27,308][44959] Updated weights for policy 1, policy_version 52310 (0.0009) [2023-10-12 22:11:27,681][44959] Updated weights for policy 1, policy_version 52320 (0.0009) [2023-10-12 22:11:29,425][44958] Updated weights for policy 0, policy_version 52040 (0.0009) [2023-10-12 22:11:29,801][44958] Updated weights for policy 0, policy_version 52050 (0.0010) [2023-10-12 22:11:30,166][44958] Updated weights for policy 0, policy_version 52060 (0.0009) [2023-10-12 22:11:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 106889216. Throughput: 0: 1653.2, 1: 1653.9. Samples: 26734220. Policy #0 lag: (min: 8.0, avg: 36.6, max: 40.0) [2023-10-12 22:11:31,443][43579] Avg episode reward: [(0, '276.640'), (1, '279.270')] [2023-10-12 22:11:31,871][44959] Updated weights for policy 1, policy_version 52330 (0.0010) [2023-10-12 22:11:32,242][44959] Updated weights for policy 1, policy_version 52340 (0.0009) [2023-10-12 22:11:32,615][44959] Updated weights for policy 1, policy_version 52350 (0.0007) [2023-10-12 22:11:34,320][44958] Updated weights for policy 0, policy_version 52070 (0.0009) [2023-10-12 22:11:34,699][44958] Updated weights for policy 0, policy_version 52080 (0.0009) [2023-10-12 22:11:35,064][44958] Updated weights for policy 0, policy_version 52090 (0.0009) [2023-10-12 22:11:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 106954752. Throughput: 0: 1649.1, 1: 1652.9. Samples: 26744132. Policy #0 lag: (min: 8.0, avg: 36.6, max: 40.0) [2023-10-12 22:11:36,443][43579] Avg episode reward: [(0, '274.540'), (1, '280.390')] [2023-10-12 22:11:36,903][44959] Updated weights for policy 1, policy_version 52360 (0.0008) [2023-10-12 22:11:37,280][44959] Updated weights for policy 1, policy_version 52370 (0.0009) [2023-10-12 22:11:37,649][44959] Updated weights for policy 1, policy_version 52380 (0.0009) [2023-10-12 22:11:39,097][44958] Updated weights for policy 0, policy_version 52100 (0.0008) [2023-10-12 22:11:39,466][44958] Updated weights for policy 0, policy_version 52110 (0.0008) [2023-10-12 22:11:39,841][44958] Updated weights for policy 0, policy_version 52120 (0.0010) [2023-10-12 22:11:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107020288. Throughput: 0: 1641.1, 1: 1652.4. Samples: 26763418. Policy #0 lag: (min: 8.0, avg: 36.6, max: 40.0) [2023-10-12 22:11:41,444][43579] Avg episode reward: [(0, '277.390'), (1, '287.040')] [2023-10-12 22:11:41,640][44959] Updated weights for policy 1, policy_version 52390 (0.0007) [2023-10-12 22:11:42,007][44959] Updated weights for policy 1, policy_version 52400 (0.0009) [2023-10-12 22:11:42,380][44959] Updated weights for policy 1, policy_version 52410 (0.0010) [2023-10-12 22:11:44,070][44958] Updated weights for policy 0, policy_version 52130 (0.0007) [2023-10-12 22:11:44,445][44958] Updated weights for policy 0, policy_version 52140 (0.0008) [2023-10-12 22:11:44,810][44958] Updated weights for policy 0, policy_version 52150 (0.0008) [2023-10-12 22:11:45,180][44958] Updated weights for policy 0, policy_version 52160 (0.0010) [2023-10-12 22:11:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107085824. Throughput: 0: 1653.0, 1: 1652.9. Samples: 26783624. Policy #0 lag: (min: 8.0, avg: 36.6, max: 40.0) [2023-10-12 22:11:46,444][43579] Avg episode reward: [(0, '278.840'), (1, '286.750')] [2023-10-12 22:11:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000052160_53411840.pth... [2023-10-12 22:11:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000050624_51838976.pth [2023-10-12 22:11:46,601][44959] Updated weights for policy 1, policy_version 52420 (0.0009) [2023-10-12 22:11:46,978][44959] Updated weights for policy 1, policy_version 52430 (0.0009) [2023-10-12 22:11:47,343][44959] Updated weights for policy 1, policy_version 52440 (0.0008) [2023-10-12 22:11:47,644][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000052448_53706752.pth... [2023-10-12 22:11:47,683][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000050880_52101120.pth [2023-10-12 22:11:49,391][44958] Updated weights for policy 0, policy_version 52170 (0.0007) [2023-10-12 22:11:49,762][44958] Updated weights for policy 0, policy_version 52180 (0.0007) [2023-10-12 22:11:50,138][44958] Updated weights for policy 0, policy_version 52190 (0.0011) [2023-10-12 22:11:51,377][44959] Updated weights for policy 1, policy_version 52450 (0.0009) [2023-10-12 22:11:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107151360. Throughput: 0: 1645.2, 1: 1657.1. Samples: 26793506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:11:51,444][43579] Avg episode reward: [(0, '278.900'), (1, '285.230')] [2023-10-12 22:11:51,748][44959] Updated weights for policy 1, policy_version 52460 (0.0007) [2023-10-12 22:11:52,111][44959] Updated weights for policy 1, policy_version 52470 (0.0008) [2023-10-12 22:11:52,481][44959] Updated weights for policy 1, policy_version 52480 (0.0009) [2023-10-12 22:11:54,352][44958] Updated weights for policy 0, policy_version 52200 (0.0009) [2023-10-12 22:11:54,724][44958] Updated weights for policy 0, policy_version 52210 (0.0008) [2023-10-12 22:11:55,096][44958] Updated weights for policy 0, policy_version 52220 (0.0008) [2023-10-12 22:11:56,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107216896. Throughput: 0: 1644.3, 1: 1657.7. Samples: 26813054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:11:56,443][43579] Avg episode reward: [(0, '279.350'), (1, '290.720')] [2023-10-12 22:11:56,845][44959] Updated weights for policy 1, policy_version 52490 (0.0008) [2023-10-12 22:11:57,211][44959] Updated weights for policy 1, policy_version 52500 (0.0010) [2023-10-12 22:11:57,578][44959] Updated weights for policy 1, policy_version 52510 (0.0010) [2023-10-12 22:11:59,136][44958] Updated weights for policy 0, policy_version 52230 (0.0009) [2023-10-12 22:11:59,506][44958] Updated weights for policy 0, policy_version 52240 (0.0008) [2023-10-12 22:11:59,885][44958] Updated weights for policy 0, policy_version 52250 (0.0010) [2023-10-12 22:12:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107282432. Throughput: 0: 1654.6, 1: 1657.7. Samples: 26833298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:01,443][43579] Avg episode reward: [(0, '279.630'), (1, '289.150')] [2023-10-12 22:12:01,527][44959] Updated weights for policy 1, policy_version 52520 (0.0009) [2023-10-12 22:12:01,902][44959] Updated weights for policy 1, policy_version 52530 (0.0010) [2023-10-12 22:12:02,268][44959] Updated weights for policy 1, policy_version 52540 (0.0008) [2023-10-12 22:12:04,093][44958] Updated weights for policy 0, policy_version 52260 (0.0008) [2023-10-12 22:12:04,473][44958] Updated weights for policy 0, policy_version 52270 (0.0009) [2023-10-12 22:12:04,839][44958] Updated weights for policy 0, policy_version 52280 (0.0008) [2023-10-12 22:12:06,309][44959] Updated weights for policy 1, policy_version 52550 (0.0009) [2023-10-12 22:12:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107347968. Throughput: 0: 1642.9, 1: 1662.9. Samples: 26843210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:06,443][43579] Avg episode reward: [(0, '275.730'), (1, '283.840')] [2023-10-12 22:12:06,682][44959] Updated weights for policy 1, policy_version 52560 (0.0007) [2023-10-12 22:12:07,044][44959] Updated weights for policy 1, policy_version 52570 (0.0007) [2023-10-12 22:12:08,856][44958] Updated weights for policy 0, policy_version 52290 (0.0009) [2023-10-12 22:12:09,229][44958] Updated weights for policy 0, policy_version 52300 (0.0010) [2023-10-12 22:12:09,596][44958] Updated weights for policy 0, policy_version 52310 (0.0007) [2023-10-12 22:12:09,969][44958] Updated weights for policy 0, policy_version 52320 (0.0009) [2023-10-12 22:12:11,176][44959] Updated weights for policy 1, policy_version 52580 (0.0007) [2023-10-12 22:12:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107413504. Throughput: 0: 1637.1, 1: 1668.0. Samples: 26862946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:11,444][43579] Avg episode reward: [(0, '273.410'), (1, '282.230')] [2023-10-12 22:12:11,541][44959] Updated weights for policy 1, policy_version 52590 (0.0007) [2023-10-12 22:12:11,899][44959] Updated weights for policy 1, policy_version 52600 (0.0008) [2023-10-12 22:12:14,259][44958] Updated weights for policy 0, policy_version 52330 (0.0009) [2023-10-12 22:12:14,635][44958] Updated weights for policy 0, policy_version 52340 (0.0009) [2023-10-12 22:12:15,013][44958] Updated weights for policy 0, policy_version 52350 (0.0007) [2023-10-12 22:12:16,146][44959] Updated weights for policy 1, policy_version 52610 (0.0009) [2023-10-12 22:12:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107479040. Throughput: 0: 1646.4, 1: 1666.8. Samples: 26883314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:16,443][43579] Avg episode reward: [(0, '276.190'), (1, '279.210')] [2023-10-12 22:12:16,575][44959] Updated weights for policy 1, policy_version 52620 (0.0010) [2023-10-12 22:12:16,938][44959] Updated weights for policy 1, policy_version 52630 (0.0010) [2023-10-12 22:12:17,303][44959] Updated weights for policy 1, policy_version 52640 (0.0010) [2023-10-12 22:12:18,853][44958] Updated weights for policy 0, policy_version 52360 (0.0008) [2023-10-12 22:12:19,232][44958] Updated weights for policy 0, policy_version 52370 (0.0007) [2023-10-12 22:12:19,609][44958] Updated weights for policy 0, policy_version 52380 (0.0010) [2023-10-12 22:12:21,313][44959] Updated weights for policy 1, policy_version 52650 (0.0009) [2023-10-12 22:12:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107544576. Throughput: 0: 1638.1, 1: 1666.3. Samples: 26892830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:21,444][43579] Avg episode reward: [(0, '272.250'), (1, '271.010')] [2023-10-12 22:12:21,679][44959] Updated weights for policy 1, policy_version 52660 (0.0007) [2023-10-12 22:12:22,057][44959] Updated weights for policy 1, policy_version 52670 (0.0007) [2023-10-12 22:12:23,802][44958] Updated weights for policy 0, policy_version 52390 (0.0009) [2023-10-12 22:12:24,173][44958] Updated weights for policy 0, policy_version 52400 (0.0008) [2023-10-12 22:12:24,548][44958] Updated weights for policy 0, policy_version 52410 (0.0007) [2023-10-12 22:12:26,259][44959] Updated weights for policy 1, policy_version 52680 (0.0008) [2023-10-12 22:12:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107610112. Throughput: 0: 1651.8, 1: 1663.6. Samples: 26912610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:26,444][43579] Avg episode reward: [(0, '270.550'), (1, '270.100')] [2023-10-12 22:12:26,631][44959] Updated weights for policy 1, policy_version 52690 (0.0010) [2023-10-12 22:12:27,002][44959] Updated weights for policy 1, policy_version 52700 (0.0009) [2023-10-12 22:12:28,892][44958] Updated weights for policy 0, policy_version 52420 (0.0007) [2023-10-12 22:12:29,273][44958] Updated weights for policy 0, policy_version 52430 (0.0010) [2023-10-12 22:12:29,644][44958] Updated weights for policy 0, policy_version 52440 (0.0009) [2023-10-12 22:12:31,038][44959] Updated weights for policy 1, policy_version 52710 (0.0010) [2023-10-12 22:12:31,413][44959] Updated weights for policy 1, policy_version 52720 (0.0009) [2023-10-12 22:12:31,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107675648. Throughput: 0: 1653.5, 1: 1653.5. Samples: 26932436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:31,443][43579] Avg episode reward: [(0, '268.830'), (1, '271.990')] [2023-10-12 22:12:31,785][44959] Updated weights for policy 1, policy_version 52730 (0.0008) [2023-10-12 22:12:33,695][44958] Updated weights for policy 0, policy_version 52450 (0.0010) [2023-10-12 22:12:34,061][44958] Updated weights for policy 0, policy_version 52460 (0.0010) [2023-10-12 22:12:34,437][44958] Updated weights for policy 0, policy_version 52470 (0.0007) [2023-10-12 22:12:34,809][44958] Updated weights for policy 0, policy_version 52480 (0.0007) [2023-10-12 22:12:36,206][44959] Updated weights for policy 1, policy_version 52740 (0.0007) [2023-10-12 22:12:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107741184. Throughput: 0: 1648.2, 1: 1658.9. Samples: 26942324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:36,443][43579] Avg episode reward: [(0, '274.360'), (1, '273.710')] [2023-10-12 22:12:36,578][44959] Updated weights for policy 1, policy_version 52750 (0.0008) [2023-10-12 22:12:36,946][44959] Updated weights for policy 1, policy_version 52760 (0.0008) [2023-10-12 22:12:38,967][44958] Updated weights for policy 0, policy_version 52490 (0.0008) [2023-10-12 22:12:39,337][44958] Updated weights for policy 0, policy_version 52500 (0.0008) [2023-10-12 22:12:39,717][44958] Updated weights for policy 0, policy_version 52510 (0.0010) [2023-10-12 22:12:41,336][44959] Updated weights for policy 1, policy_version 52770 (0.0008) [2023-10-12 22:12:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 107806720. Throughput: 0: 1650.9, 1: 1651.1. Samples: 26961644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:41,444][43579] Avg episode reward: [(0, '267.360'), (1, '269.650')] [2023-10-12 22:12:41,714][44959] Updated weights for policy 1, policy_version 52780 (0.0011) [2023-10-12 22:12:42,079][44959] Updated weights for policy 1, policy_version 52790 (0.0009) [2023-10-12 22:12:42,444][44959] Updated weights for policy 1, policy_version 52800 (0.0008) [2023-10-12 22:12:43,711][44958] Updated weights for policy 0, policy_version 52520 (0.0008) [2023-10-12 22:12:44,080][44958] Updated weights for policy 0, policy_version 52530 (0.0008) [2023-10-12 22:12:44,461][44958] Updated weights for policy 0, policy_version 52540 (0.0007) [2023-10-12 22:12:46,330][44959] Updated weights for policy 1, policy_version 52810 (0.0010) [2023-10-12 22:12:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 107872256. Throughput: 0: 1659.9, 1: 1645.7. Samples: 26982050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:46,443][43579] Avg episode reward: [(0, '264.940'), (1, '276.190')] [2023-10-12 22:12:46,690][44959] Updated weights for policy 1, policy_version 52820 (0.0009) [2023-10-12 22:12:47,055][44959] Updated weights for policy 1, policy_version 52830 (0.0009) [2023-10-12 22:12:48,533][44958] Updated weights for policy 0, policy_version 52550 (0.0009) [2023-10-12 22:12:48,902][44958] Updated weights for policy 0, policy_version 52560 (0.0010) [2023-10-12 22:12:49,286][44958] Updated weights for policy 0, policy_version 52570 (0.0010) [2023-10-12 22:12:51,303][44959] Updated weights for policy 1, policy_version 52840 (0.0008) [2023-10-12 22:12:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 107937792. Throughput: 0: 1649.6, 1: 1643.4. Samples: 26991398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:51,443][43579] Avg episode reward: [(0, '265.340'), (1, '277.170')] [2023-10-12 22:12:51,680][44959] Updated weights for policy 1, policy_version 52850 (0.0007) [2023-10-12 22:12:52,036][44959] Updated weights for policy 1, policy_version 52860 (0.0007) [2023-10-12 22:12:53,735][44958] Updated weights for policy 0, policy_version 52580 (0.0008) [2023-10-12 22:12:54,116][44958] Updated weights for policy 0, policy_version 52590 (0.0009) [2023-10-12 22:12:54,487][44958] Updated weights for policy 0, policy_version 52600 (0.0010) [2023-10-12 22:12:56,304][44959] Updated weights for policy 1, policy_version 52870 (0.0008) [2023-10-12 22:12:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108003328. Throughput: 0: 1656.0, 1: 1637.4. Samples: 27011148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:12:56,443][43579] Avg episode reward: [(0, '274.540'), (1, '275.430')] [2023-10-12 22:12:56,676][44959] Updated weights for policy 1, policy_version 52880 (0.0008) [2023-10-12 22:12:57,049][44959] Updated weights for policy 1, policy_version 52890 (0.0007) [2023-10-12 22:12:58,475][44958] Updated weights for policy 0, policy_version 52610 (0.0010) [2023-10-12 22:12:58,847][44958] Updated weights for policy 0, policy_version 52620 (0.0007) [2023-10-12 22:12:59,208][44958] Updated weights for policy 0, policy_version 52630 (0.0008) [2023-10-12 22:12:59,588][44958] Updated weights for policy 0, policy_version 52640 (0.0007) [2023-10-12 22:13:01,103][44959] Updated weights for policy 1, policy_version 52900 (0.0007) [2023-10-12 22:13:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108068864. Throughput: 0: 1657.6, 1: 1630.7. Samples: 27031286. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-12 22:13:01,443][43579] Avg episode reward: [(0, '272.670'), (1, '278.850')] [2023-10-12 22:13:01,493][44959] Updated weights for policy 1, policy_version 52910 (0.0009) [2023-10-12 22:13:01,861][44959] Updated weights for policy 1, policy_version 52920 (0.0009) [2023-10-12 22:13:03,858][44958] Updated weights for policy 0, policy_version 52650 (0.0011) [2023-10-12 22:13:04,226][44958] Updated weights for policy 0, policy_version 52660 (0.0009) [2023-10-12 22:13:04,605][44958] Updated weights for policy 0, policy_version 52670 (0.0008) [2023-10-12 22:13:06,144][44959] Updated weights for policy 1, policy_version 52930 (0.0009) [2023-10-12 22:13:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108134400. Throughput: 0: 1650.7, 1: 1634.2. Samples: 27040650. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-12 22:13:06,443][43579] Avg episode reward: [(0, '271.850'), (1, '279.010')] [2023-10-12 22:13:06,504][44959] Updated weights for policy 1, policy_version 52940 (0.0008) [2023-10-12 22:13:06,885][44959] Updated weights for policy 1, policy_version 52950 (0.0009) [2023-10-12 22:13:07,250][44959] Updated weights for policy 1, policy_version 52960 (0.0008) [2023-10-12 22:13:08,875][44958] Updated weights for policy 0, policy_version 52680 (0.0009) [2023-10-12 22:13:09,248][44958] Updated weights for policy 0, policy_version 52690 (0.0007) [2023-10-12 22:13:09,626][44958] Updated weights for policy 0, policy_version 52700 (0.0007) [2023-10-12 22:13:11,420][44959] Updated weights for policy 1, policy_version 52970 (0.0009) [2023-10-12 22:13:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108199936. Throughput: 0: 1642.7, 1: 1630.5. Samples: 27059902. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-12 22:13:11,443][43579] Avg episode reward: [(0, '273.580'), (1, '281.670')] [2023-10-12 22:13:11,794][44959] Updated weights for policy 1, policy_version 52980 (0.0010) [2023-10-12 22:13:12,166][44959] Updated weights for policy 1, policy_version 52990 (0.0011) [2023-10-12 22:13:13,721][44958] Updated weights for policy 0, policy_version 52710 (0.0009) [2023-10-12 22:13:14,108][44958] Updated weights for policy 0, policy_version 52720 (0.0009) [2023-10-12 22:13:14,478][44958] Updated weights for policy 0, policy_version 52730 (0.0008) [2023-10-12 22:13:16,173][44959] Updated weights for policy 1, policy_version 53000 (0.0009) [2023-10-12 22:13:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108265472. Throughput: 0: 1641.3, 1: 1636.8. Samples: 27079952. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-12 22:13:16,443][43579] Avg episode reward: [(0, '275.620'), (1, '286.620')] [2023-10-12 22:13:16,541][44959] Updated weights for policy 1, policy_version 53010 (0.0008) [2023-10-12 22:13:16,903][44959] Updated weights for policy 1, policy_version 53020 (0.0008) [2023-10-12 22:13:18,771][44958] Updated weights for policy 0, policy_version 52740 (0.0008) [2023-10-12 22:13:19,139][44958] Updated weights for policy 0, policy_version 52750 (0.0009) [2023-10-12 22:13:19,510][44958] Updated weights for policy 0, policy_version 52760 (0.0009) [2023-10-12 22:13:20,961][44959] Updated weights for policy 1, policy_version 53030 (0.0007) [2023-10-12 22:13:21,338][44959] Updated weights for policy 1, policy_version 53040 (0.0009) [2023-10-12 22:13:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108331008. Throughput: 0: 1638.0, 1: 1638.9. Samples: 27089786. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-12 22:13:21,444][43579] Avg episode reward: [(0, '279.930'), (1, '287.170')] [2023-10-12 22:13:21,711][44959] Updated weights for policy 1, policy_version 53050 (0.0009) [2023-10-12 22:13:23,933][44958] Updated weights for policy 0, policy_version 52770 (0.0009) [2023-10-12 22:13:24,309][44958] Updated weights for policy 0, policy_version 52780 (0.0009) [2023-10-12 22:13:24,679][44958] Updated weights for policy 0, policy_version 52790 (0.0008) [2023-10-12 22:13:25,056][44958] Updated weights for policy 0, policy_version 52800 (0.0008) [2023-10-12 22:13:26,209][44959] Updated weights for policy 1, policy_version 53060 (0.0007) [2023-10-12 22:13:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108396544. Throughput: 0: 1634.1, 1: 1641.5. Samples: 27109042. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-12 22:13:26,443][43579] Avg episode reward: [(0, '276.950'), (1, '287.670')] [2023-10-12 22:13:26,581][44959] Updated weights for policy 1, policy_version 53070 (0.0010) [2023-10-12 22:13:26,942][44959] Updated weights for policy 1, policy_version 53080 (0.0008) [2023-10-12 22:13:29,134][44958] Updated weights for policy 0, policy_version 52810 (0.0007) [2023-10-12 22:13:29,502][44958] Updated weights for policy 0, policy_version 52820 (0.0007) [2023-10-12 22:13:29,883][44958] Updated weights for policy 0, policy_version 52830 (0.0008) [2023-10-12 22:13:31,183][44959] Updated weights for policy 1, policy_version 53090 (0.0008) [2023-10-12 22:13:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108462080. Throughput: 0: 1626.0, 1: 1643.6. Samples: 27129186. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-12 22:13:31,443][43579] Avg episode reward: [(0, '279.650'), (1, '288.470')] [2023-10-12 22:13:31,552][44959] Updated weights for policy 1, policy_version 53100 (0.0010) [2023-10-12 22:13:31,930][44959] Updated weights for policy 1, policy_version 53110 (0.0007) [2023-10-12 22:13:32,296][44959] Updated weights for policy 1, policy_version 53120 (0.0007) [2023-10-12 22:13:34,276][44958] Updated weights for policy 0, policy_version 52840 (0.0007) [2023-10-12 22:13:34,651][44958] Updated weights for policy 0, policy_version 52850 (0.0008) [2023-10-12 22:13:35,021][44958] Updated weights for policy 0, policy_version 52860 (0.0010) [2023-10-12 22:13:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108527616. Throughput: 0: 1636.4, 1: 1643.3. Samples: 27138986. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:13:36,444][43579] Avg episode reward: [(0, '280.360'), (1, '292.280')] [2023-10-12 22:13:36,501][44959] Updated weights for policy 1, policy_version 53130 (0.0008) [2023-10-12 22:13:36,872][44959] Updated weights for policy 1, policy_version 53140 (0.0011) [2023-10-12 22:13:37,243][44959] Updated weights for policy 1, policy_version 53150 (0.0010) [2023-10-12 22:13:37,312][44583] Saving new best policy, reward=292.280! [2023-10-12 22:13:38,928][44958] Updated weights for policy 0, policy_version 52870 (0.0008) [2023-10-12 22:13:39,305][44958] Updated weights for policy 0, policy_version 52880 (0.0010) [2023-10-12 22:13:39,685][44958] Updated weights for policy 0, policy_version 52890 (0.0009) [2023-10-12 22:13:41,341][44959] Updated weights for policy 1, policy_version 53160 (0.0008) [2023-10-12 22:13:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108593152. Throughput: 0: 1633.1, 1: 1635.3. Samples: 27158224. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:13:41,443][43579] Avg episode reward: [(0, '284.940'), (1, '288.250')] [2023-10-12 22:13:41,715][44959] Updated weights for policy 1, policy_version 53170 (0.0007) [2023-10-12 22:13:42,074][44959] Updated weights for policy 1, policy_version 53180 (0.0007) [2023-10-12 22:13:44,026][44958] Updated weights for policy 0, policy_version 52900 (0.0009) [2023-10-12 22:13:44,398][44958] Updated weights for policy 0, policy_version 52910 (0.0009) [2023-10-12 22:13:44,778][44958] Updated weights for policy 0, policy_version 52920 (0.0009) [2023-10-12 22:13:46,417][44959] Updated weights for policy 1, policy_version 53190 (0.0009) [2023-10-12 22:13:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108658688. Throughput: 0: 1627.2, 1: 1644.2. Samples: 27178502. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:13:46,443][43579] Avg episode reward: [(0, '282.090'), (1, '288.490')] [2023-10-12 22:13:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000052928_54198272.pth... [2023-10-12 22:13:46,490][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000051392_52625408.pth [2023-10-12 22:13:46,798][44959] Updated weights for policy 1, policy_version 53200 (0.0009) [2023-10-12 22:13:47,167][44959] Updated weights for policy 1, policy_version 53210 (0.0010) [2023-10-12 22:13:47,385][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000053216_54493184.pth... [2023-10-12 22:13:47,424][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000051648_52887552.pth [2023-10-12 22:13:48,843][44958] Updated weights for policy 0, policy_version 52930 (0.0008) [2023-10-12 22:13:49,213][44958] Updated weights for policy 0, policy_version 52940 (0.0008) [2023-10-12 22:13:49,579][44958] Updated weights for policy 0, policy_version 52950 (0.0008) [2023-10-12 22:13:49,960][44958] Updated weights for policy 0, policy_version 52960 (0.0009) [2023-10-12 22:13:51,360][44959] Updated weights for policy 1, policy_version 53220 (0.0009) [2023-10-12 22:13:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108724224. Throughput: 0: 1635.0, 1: 1638.6. Samples: 27187962. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:13:51,443][43579] Avg episode reward: [(0, '278.480'), (1, '287.360')] [2023-10-12 22:13:51,724][44959] Updated weights for policy 1, policy_version 53230 (0.0008) [2023-10-12 22:13:52,089][44959] Updated weights for policy 1, policy_version 53240 (0.0009) [2023-10-12 22:13:54,146][44958] Updated weights for policy 0, policy_version 52970 (0.0009) [2023-10-12 22:13:54,520][44958] Updated weights for policy 0, policy_version 52980 (0.0009) [2023-10-12 22:13:54,889][44958] Updated weights for policy 0, policy_version 52990 (0.0007) [2023-10-12 22:13:56,091][44959] Updated weights for policy 1, policy_version 53250 (0.0008) [2023-10-12 22:13:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108789760. Throughput: 0: 1639.1, 1: 1641.8. Samples: 27207542. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:13:56,444][43579] Avg episode reward: [(0, '277.330'), (1, '283.500')] [2023-10-12 22:13:56,455][44959] Updated weights for policy 1, policy_version 53260 (0.0010) [2023-10-12 22:13:56,832][44959] Updated weights for policy 1, policy_version 53270 (0.0010) [2023-10-12 22:13:57,196][44959] Updated weights for policy 1, policy_version 53280 (0.0009) [2023-10-12 22:13:59,048][44958] Updated weights for policy 0, policy_version 53000 (0.0009) [2023-10-12 22:13:59,417][44958] Updated weights for policy 0, policy_version 53010 (0.0008) [2023-10-12 22:13:59,788][44958] Updated weights for policy 0, policy_version 53020 (0.0007) [2023-10-12 22:14:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 108855296. Throughput: 0: 1642.3, 1: 1645.5. Samples: 27227906. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:14:01,444][43579] Avg episode reward: [(0, '275.320'), (1, '282.390')] [2023-10-12 22:14:01,538][44959] Updated weights for policy 1, policy_version 53290 (0.0007) [2023-10-12 22:14:01,905][44959] Updated weights for policy 1, policy_version 53300 (0.0009) [2023-10-12 22:14:02,276][44959] Updated weights for policy 1, policy_version 53310 (0.0009) [2023-10-12 22:14:03,842][44958] Updated weights for policy 0, policy_version 53030 (0.0008) [2023-10-12 22:14:04,211][44958] Updated weights for policy 0, policy_version 53040 (0.0007) [2023-10-12 22:14:04,588][44958] Updated weights for policy 0, policy_version 53050 (0.0009) [2023-10-12 22:14:06,273][44959] Updated weights for policy 1, policy_version 53320 (0.0009) [2023-10-12 22:14:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108920832. Throughput: 0: 1645.1, 1: 1637.1. Samples: 27237482. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-12 22:14:06,443][43579] Avg episode reward: [(0, '275.630'), (1, '282.600')] [2023-10-12 22:14:06,650][44959] Updated weights for policy 1, policy_version 53330 (0.0009) [2023-10-12 22:14:07,008][44959] Updated weights for policy 1, policy_version 53340 (0.0009) [2023-10-12 22:14:08,801][44958] Updated weights for policy 0, policy_version 53060 (0.0008) [2023-10-12 22:14:09,163][44958] Updated weights for policy 0, policy_version 53070 (0.0009) [2023-10-12 22:14:09,541][44958] Updated weights for policy 0, policy_version 53080 (0.0007) [2023-10-12 22:14:11,142][44959] Updated weights for policy 1, policy_version 53350 (0.0007) [2023-10-12 22:14:11,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 108986368. Throughput: 0: 1648.6, 1: 1644.6. Samples: 27257238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:14:11,443][43579] Avg episode reward: [(0, '275.790'), (1, '276.350')] [2023-10-12 22:14:11,515][44959] Updated weights for policy 1, policy_version 53360 (0.0007) [2023-10-12 22:14:11,883][44959] Updated weights for policy 1, policy_version 53370 (0.0008) [2023-10-12 22:14:13,582][44958] Updated weights for policy 0, policy_version 53090 (0.0008) [2023-10-12 22:14:13,959][44958] Updated weights for policy 0, policy_version 53100 (0.0010) [2023-10-12 22:14:14,336][44958] Updated weights for policy 0, policy_version 53110 (0.0010) [2023-10-12 22:14:14,709][44958] Updated weights for policy 0, policy_version 53120 (0.0009) [2023-10-12 22:14:15,739][44959] Updated weights for policy 1, policy_version 53380 (0.0009) [2023-10-12 22:14:16,097][44959] Updated weights for policy 1, policy_version 53390 (0.0010) [2023-10-12 22:14:16,443][43579] Fps is (10 sec: 13106.6, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 109051904. Throughput: 0: 1649.0, 1: 1642.6. Samples: 27277312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:14:16,444][43579] Avg episode reward: [(0, '276.750'), (1, '272.560')] [2023-10-12 22:14:16,470][44959] Updated weights for policy 1, policy_version 53400 (0.0010) [2023-10-12 22:14:18,901][44958] Updated weights for policy 0, policy_version 53130 (0.0008) [2023-10-12 22:14:19,282][44958] Updated weights for policy 0, policy_version 53140 (0.0009) [2023-10-12 22:14:19,650][44958] Updated weights for policy 0, policy_version 53150 (0.0009) [2023-10-12 22:14:20,745][44959] Updated weights for policy 1, policy_version 53410 (0.0008) [2023-10-12 22:14:21,114][44959] Updated weights for policy 1, policy_version 53420 (0.0009) [2023-10-12 22:14:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 109117440. Throughput: 0: 1642.8, 1: 1652.6. Samples: 27287282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:14:21,443][43579] Avg episode reward: [(0, '278.110'), (1, '271.380')] [2023-10-12 22:14:21,497][44959] Updated weights for policy 1, policy_version 53430 (0.0010) [2023-10-12 22:14:21,867][44959] Updated weights for policy 1, policy_version 53440 (0.0010) [2023-10-12 22:14:23,823][44958] Updated weights for policy 0, policy_version 53160 (0.0010) [2023-10-12 22:14:24,189][44958] Updated weights for policy 0, policy_version 53170 (0.0010) [2023-10-12 22:14:24,563][44958] Updated weights for policy 0, policy_version 53180 (0.0010) [2023-10-12 22:14:26,039][44959] Updated weights for policy 1, policy_version 53450 (0.0010) [2023-10-12 22:14:26,408][44959] Updated weights for policy 1, policy_version 53460 (0.0008) [2023-10-12 22:14:26,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 109182976. Throughput: 0: 1643.2, 1: 1652.0. Samples: 27306508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:14:26,444][43579] Avg episode reward: [(0, '283.040'), (1, '267.240')] [2023-10-12 22:14:26,774][44959] Updated weights for policy 1, policy_version 53470 (0.0009) [2023-10-12 22:14:28,689][44958] Updated weights for policy 0, policy_version 53190 (0.0009) [2023-10-12 22:14:29,060][44958] Updated weights for policy 0, policy_version 53200 (0.0010) [2023-10-12 22:14:29,439][44958] Updated weights for policy 0, policy_version 53210 (0.0008) [2023-10-12 22:14:31,114][44959] Updated weights for policy 1, policy_version 53480 (0.0010) [2023-10-12 22:14:31,443][43579] Fps is (10 sec: 13106.6, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 109248512. Throughput: 0: 1650.1, 1: 1647.2. Samples: 27326880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:14:31,444][43579] Avg episode reward: [(0, '279.260'), (1, '268.490')] [2023-10-12 22:14:31,478][44959] Updated weights for policy 1, policy_version 53490 (0.0008) [2023-10-12 22:14:31,854][44959] Updated weights for policy 1, policy_version 53500 (0.0009) [2023-10-12 22:14:33,661][44958] Updated weights for policy 0, policy_version 53220 (0.0010) [2023-10-12 22:14:34,031][44958] Updated weights for policy 0, policy_version 53230 (0.0011) [2023-10-12 22:14:34,398][44958] Updated weights for policy 0, policy_version 53240 (0.0008) [2023-10-12 22:14:35,794][44959] Updated weights for policy 1, policy_version 53510 (0.0009) [2023-10-12 22:14:36,175][44959] Updated weights for policy 1, policy_version 53520 (0.0009) [2023-10-12 22:14:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 109314048. Throughput: 0: 1643.6, 1: 1661.3. Samples: 27336682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:14:36,443][43579] Avg episode reward: [(0, '278.040'), (1, '269.070')] [2023-10-12 22:14:36,534][44959] Updated weights for policy 1, policy_version 53530 (0.0008) [2023-10-12 22:14:38,628][44958] Updated weights for policy 0, policy_version 53250 (0.0007) [2023-10-12 22:14:39,003][44958] Updated weights for policy 0, policy_version 53260 (0.0007) [2023-10-12 22:14:39,378][44958] Updated weights for policy 0, policy_version 53270 (0.0008) [2023-10-12 22:14:39,754][44958] Updated weights for policy 0, policy_version 53280 (0.0007) [2023-10-12 22:14:40,718][44959] Updated weights for policy 1, policy_version 53540 (0.0009) [2023-10-12 22:14:41,081][44959] Updated weights for policy 1, policy_version 53550 (0.0007) [2023-10-12 22:14:41,442][43579] Fps is (10 sec: 13107.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 109379584. Throughput: 0: 1644.4, 1: 1659.3. Samples: 27356208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:14:41,443][43579] Avg episode reward: [(0, '272.530'), (1, '272.370')] [2023-10-12 22:14:41,455][44959] Updated weights for policy 1, policy_version 53560 (0.0008) [2023-10-12 22:14:43,786][44958] Updated weights for policy 0, policy_version 53290 (0.0009) [2023-10-12 22:14:44,157][44958] Updated weights for policy 0, policy_version 53300 (0.0008) [2023-10-12 22:14:44,524][44958] Updated weights for policy 0, policy_version 53310 (0.0008) [2023-10-12 22:14:45,402][44959] Updated weights for policy 1, policy_version 53570 (0.0011) [2023-10-12 22:14:45,764][44959] Updated weights for policy 1, policy_version 53580 (0.0009) [2023-10-12 22:14:46,131][44959] Updated weights for policy 1, policy_version 53590 (0.0010) [2023-10-12 22:14:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 109445120. Throughput: 0: 1643.9, 1: 1646.5. Samples: 27375976. Policy #0 lag: (min: 10.0, avg: 14.3, max: 42.0) [2023-10-12 22:14:46,443][43579] Avg episode reward: [(0, '272.200'), (1, '277.850')] [2023-10-12 22:14:46,500][44959] Updated weights for policy 1, policy_version 53600 (0.0008) [2023-10-12 22:14:48,710][44958] Updated weights for policy 0, policy_version 53320 (0.0008) [2023-10-12 22:14:49,079][44958] Updated weights for policy 0, policy_version 53330 (0.0007) [2023-10-12 22:14:49,449][44958] Updated weights for policy 0, policy_version 53340 (0.0007) [2023-10-12 22:14:50,578][44959] Updated weights for policy 1, policy_version 53610 (0.0009) [2023-10-12 22:14:50,954][44959] Updated weights for policy 1, policy_version 53620 (0.0008) [2023-10-12 22:14:51,320][44959] Updated weights for policy 1, policy_version 53630 (0.0008) [2023-10-12 22:14:51,442][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 109543424. Throughput: 0: 1635.1, 1: 1660.9. Samples: 27385800. Policy #0 lag: (min: 10.0, avg: 14.3, max: 42.0) [2023-10-12 22:14:51,443][43579] Avg episode reward: [(0, '271.530'), (1, '275.430')] [2023-10-12 22:14:53,707][44958] Updated weights for policy 0, policy_version 53350 (0.0008) [2023-10-12 22:14:54,076][44958] Updated weights for policy 0, policy_version 53360 (0.0008) [2023-10-12 22:14:54,440][44958] Updated weights for policy 0, policy_version 53370 (0.0007) [2023-10-12 22:14:55,249][44959] Updated weights for policy 1, policy_version 53640 (0.0007) [2023-10-12 22:14:55,624][44959] Updated weights for policy 1, policy_version 53650 (0.0008) [2023-10-12 22:14:55,999][44959] Updated weights for policy 1, policy_version 53660 (0.0007) [2023-10-12 22:14:56,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 109608960. Throughput: 0: 1636.5, 1: 1660.1. Samples: 27405588. Policy #0 lag: (min: 10.0, avg: 14.3, max: 42.0) [2023-10-12 22:14:56,444][43579] Avg episode reward: [(0, '262.910'), (1, '278.540')] [2023-10-12 22:14:58,539][44958] Updated weights for policy 0, policy_version 53380 (0.0009) [2023-10-12 22:14:58,924][44958] Updated weights for policy 0, policy_version 53390 (0.0010) [2023-10-12 22:14:59,304][44958] Updated weights for policy 0, policy_version 53400 (0.0010) [2023-10-12 22:15:00,201][44959] Updated weights for policy 1, policy_version 53670 (0.0010) [2023-10-12 22:15:00,584][44959] Updated weights for policy 1, policy_version 53680 (0.0010) [2023-10-12 22:15:00,960][44959] Updated weights for policy 1, policy_version 53690 (0.0011) [2023-10-12 22:15:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 109674496. Throughput: 0: 1638.8, 1: 1642.2. Samples: 27424956. Policy #0 lag: (min: 10.0, avg: 14.3, max: 42.0) [2023-10-12 22:15:01,444][43579] Avg episode reward: [(0, '267.320'), (1, '276.270')] [2023-10-12 22:15:03,447][44958] Updated weights for policy 0, policy_version 53410 (0.0009) [2023-10-12 22:15:03,821][44958] Updated weights for policy 0, policy_version 53420 (0.0010) [2023-10-12 22:15:04,192][44958] Updated weights for policy 0, policy_version 53430 (0.0009) [2023-10-12 22:15:04,560][44958] Updated weights for policy 0, policy_version 53440 (0.0009) [2023-10-12 22:15:05,194][44959] Updated weights for policy 1, policy_version 53700 (0.0009) [2023-10-12 22:15:05,571][44959] Updated weights for policy 1, policy_version 53710 (0.0007) [2023-10-12 22:15:05,934][44959] Updated weights for policy 1, policy_version 53720 (0.0008) [2023-10-12 22:15:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 109740032. Throughput: 0: 1632.9, 1: 1656.4. Samples: 27435302. Policy #0 lag: (min: 10.0, avg: 14.3, max: 42.0) [2023-10-12 22:15:06,444][43579] Avg episode reward: [(0, '270.800'), (1, '274.670')] [2023-10-12 22:15:08,946][44958] Updated weights for policy 0, policy_version 53450 (0.0008) [2023-10-12 22:15:09,323][44958] Updated weights for policy 0, policy_version 53460 (0.0007) [2023-10-12 22:15:09,697][44958] Updated weights for policy 0, policy_version 53470 (0.0007) [2023-10-12 22:15:10,026][44959] Updated weights for policy 1, policy_version 53730 (0.0009) [2023-10-12 22:15:10,390][44959] Updated weights for policy 1, policy_version 53740 (0.0008) [2023-10-12 22:15:10,746][44959] Updated weights for policy 1, policy_version 53750 (0.0007) [2023-10-12 22:15:11,113][44959] Updated weights for policy 1, policy_version 53760 (0.0007) [2023-10-12 22:15:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 109805568. Throughput: 0: 1639.0, 1: 1658.9. Samples: 27454912. Policy #0 lag: (min: 10.0, avg: 14.3, max: 42.0) [2023-10-12 22:15:11,443][43579] Avg episode reward: [(0, '272.520'), (1, '275.300')] [2023-10-12 22:15:13,837][44958] Updated weights for policy 0, policy_version 53480 (0.0008) [2023-10-12 22:15:14,206][44958] Updated weights for policy 0, policy_version 53490 (0.0008) [2023-10-12 22:15:14,577][44958] Updated weights for policy 0, policy_version 53500 (0.0010) [2023-10-12 22:15:15,451][44959] Updated weights for policy 1, policy_version 53770 (0.0007) [2023-10-12 22:15:15,813][44959] Updated weights for policy 1, policy_version 53780 (0.0007) [2023-10-12 22:15:16,182][44959] Updated weights for policy 1, policy_version 53790 (0.0007) [2023-10-12 22:15:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 109871104. Throughput: 0: 1638.0, 1: 1641.4. Samples: 27474452. Policy #0 lag: (min: 10.0, avg: 14.3, max: 42.0) [2023-10-12 22:15:16,444][43579] Avg episode reward: [(0, '267.800'), (1, '269.210')] [2023-10-12 22:15:18,682][44958] Updated weights for policy 0, policy_version 53510 (0.0007) [2023-10-12 22:15:19,047][44958] Updated weights for policy 0, policy_version 53520 (0.0010) [2023-10-12 22:15:19,424][44958] Updated weights for policy 0, policy_version 53530 (0.0009) [2023-10-12 22:15:20,384][44959] Updated weights for policy 1, policy_version 53800 (0.0008) [2023-10-12 22:15:20,764][44959] Updated weights for policy 1, policy_version 53810 (0.0009) [2023-10-12 22:15:21,127][44959] Updated weights for policy 1, policy_version 53820 (0.0008) [2023-10-12 22:15:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 109936640. Throughput: 0: 1636.3, 1: 1654.4. Samples: 27484764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:15:21,443][43579] Avg episode reward: [(0, '265.660'), (1, '272.230')] [2023-10-12 22:15:23,617][44958] Updated weights for policy 0, policy_version 53540 (0.0009) [2023-10-12 22:15:23,996][44958] Updated weights for policy 0, policy_version 53550 (0.0009) [2023-10-12 22:15:24,361][44958] Updated weights for policy 0, policy_version 53560 (0.0007) [2023-10-12 22:15:25,502][44959] Updated weights for policy 1, policy_version 53830 (0.0010) [2023-10-12 22:15:25,869][44959] Updated weights for policy 1, policy_version 53840 (0.0008) [2023-10-12 22:15:26,231][44959] Updated weights for policy 1, policy_version 53850 (0.0009) [2023-10-12 22:15:26,443][43579] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 109969408. Throughput: 0: 1641.0, 1: 1648.6. Samples: 27504238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:15:26,443][43579] Avg episode reward: [(0, '277.370'), (1, '269.950')] [2023-10-12 22:15:28,470][44958] Updated weights for policy 0, policy_version 53570 (0.0007) [2023-10-12 22:15:28,868][44958] Updated weights for policy 0, policy_version 53580 (0.0009) [2023-10-12 22:15:29,245][44958] Updated weights for policy 0, policy_version 53590 (0.0010) [2023-10-12 22:15:29,615][44958] Updated weights for policy 0, policy_version 53600 (0.0010) [2023-10-12 22:15:30,423][44959] Updated weights for policy 1, policy_version 53860 (0.0011) [2023-10-12 22:15:30,794][44959] Updated weights for policy 1, policy_version 53870 (0.0009) [2023-10-12 22:15:31,164][44959] Updated weights for policy 1, policy_version 53880 (0.0009) [2023-10-12 22:15:31,442][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 110034944. Throughput: 0: 1639.2, 1: 1641.0. Samples: 27523586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:15:31,443][43579] Avg episode reward: [(0, '273.460'), (1, '274.160')] [2023-10-12 22:15:33,630][44958] Updated weights for policy 0, policy_version 53610 (0.0008) [2023-10-12 22:15:34,014][44958] Updated weights for policy 0, policy_version 53620 (0.0008) [2023-10-12 22:15:34,386][44958] Updated weights for policy 0, policy_version 53630 (0.0008) [2023-10-12 22:15:35,138][44959] Updated weights for policy 1, policy_version 53890 (0.0010) [2023-10-12 22:15:35,511][44959] Updated weights for policy 1, policy_version 53900 (0.0009) [2023-10-12 22:15:35,880][44959] Updated weights for policy 1, policy_version 53910 (0.0011) [2023-10-12 22:15:36,255][44959] Updated weights for policy 1, policy_version 53920 (0.0010) [2023-10-12 22:15:36,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 110133248. Throughput: 0: 1638.9, 1: 1652.4. Samples: 27533910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:15:36,444][43579] Avg episode reward: [(0, '271.370'), (1, '270.900')] [2023-10-12 22:15:38,578][44958] Updated weights for policy 0, policy_version 53640 (0.0011) [2023-10-12 22:15:38,942][44958] Updated weights for policy 0, policy_version 53650 (0.0009) [2023-10-12 22:15:39,328][44958] Updated weights for policy 0, policy_version 53660 (0.0010) [2023-10-12 22:15:40,300][44959] Updated weights for policy 1, policy_version 53930 (0.0008) [2023-10-12 22:15:40,673][44959] Updated weights for policy 1, policy_version 53940 (0.0009) [2023-10-12 22:15:41,045][44959] Updated weights for policy 1, policy_version 53950 (0.0009) [2023-10-12 22:15:41,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 110198784. Throughput: 0: 1644.5, 1: 1645.6. Samples: 27553646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:15:41,444][43579] Avg episode reward: [(0, '272.140'), (1, '274.480')] [2023-10-12 22:15:43,488][44958] Updated weights for policy 0, policy_version 53670 (0.0008) [2023-10-12 22:15:43,858][44958] Updated weights for policy 0, policy_version 53680 (0.0007) [2023-10-12 22:15:44,236][44958] Updated weights for policy 0, policy_version 53690 (0.0010) [2023-10-12 22:15:45,220][44959] Updated weights for policy 1, policy_version 53960 (0.0008) [2023-10-12 22:15:45,594][44959] Updated weights for policy 1, policy_version 53970 (0.0011) [2023-10-12 22:15:45,969][44959] Updated weights for policy 1, policy_version 53980 (0.0010) [2023-10-12 22:15:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 110264320. Throughput: 0: 1646.4, 1: 1646.2. Samples: 27573124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:15:46,444][43579] Avg episode reward: [(0, '279.080'), (1, '276.720')] [2023-10-12 22:15:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000053984_55279616.pth... [2023-10-12 22:15:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000053696_54984704.pth... [2023-10-12 22:15:46,496][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000052448_53706752.pth [2023-10-12 22:15:46,496][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000052160_53411840.pth [2023-10-12 22:15:46,501][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000053696_54984704.pth [2023-10-12 22:15:46,502][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000053984_55279616.pth [2023-10-12 22:15:48,605][44958] Updated weights for policy 0, policy_version 53700 (0.0011) [2023-10-12 22:15:48,964][44958] Updated weights for policy 0, policy_version 53710 (0.0007) [2023-10-12 22:15:49,338][44958] Updated weights for policy 0, policy_version 53720 (0.0008) [2023-10-12 22:15:50,183][44959] Updated weights for policy 1, policy_version 53990 (0.0008) [2023-10-12 22:15:50,541][44959] Updated weights for policy 1, policy_version 54000 (0.0008) [2023-10-12 22:15:50,912][44959] Updated weights for policy 1, policy_version 54010 (0.0009) [2023-10-12 22:15:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 110329856. Throughput: 0: 1646.2, 1: 1647.6. Samples: 27583522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:15:51,443][43579] Avg episode reward: [(0, '280.370'), (1, '274.060')] [2023-10-12 22:15:53,441][44958] Updated weights for policy 0, policy_version 53730 (0.0007) [2023-10-12 22:15:53,812][44958] Updated weights for policy 0, policy_version 53740 (0.0011) [2023-10-12 22:15:54,179][44958] Updated weights for policy 0, policy_version 53750 (0.0010) [2023-10-12 22:15:54,548][44958] Updated weights for policy 0, policy_version 53760 (0.0011) [2023-10-12 22:15:55,096][44959] Updated weights for policy 1, policy_version 54020 (0.0008) [2023-10-12 22:15:55,474][44959] Updated weights for policy 1, policy_version 54030 (0.0009) [2023-10-12 22:15:55,840][44959] Updated weights for policy 1, policy_version 54040 (0.0009) [2023-10-12 22:15:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 110395392. Throughput: 0: 1647.5, 1: 1645.7. Samples: 27603110. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) [2023-10-12 22:15:56,444][43579] Avg episode reward: [(0, '279.550'), (1, '277.790')] [2023-10-12 22:15:58,793][44958] Updated weights for policy 0, policy_version 53770 (0.0008) [2023-10-12 22:15:59,166][44958] Updated weights for policy 0, policy_version 53780 (0.0008) [2023-10-12 22:15:59,546][44958] Updated weights for policy 0, policy_version 53790 (0.0007) [2023-10-12 22:15:59,984][44959] Updated weights for policy 1, policy_version 54050 (0.0008) [2023-10-12 22:16:00,357][44959] Updated weights for policy 1, policy_version 54060 (0.0009) [2023-10-12 22:16:00,733][44959] Updated weights for policy 1, policy_version 54070 (0.0009) [2023-10-12 22:16:01,105][44959] Updated weights for policy 1, policy_version 54080 (0.0010) [2023-10-12 22:16:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 110460928. Throughput: 0: 1645.9, 1: 1644.0. Samples: 27622496. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) [2023-10-12 22:16:01,443][43579] Avg episode reward: [(0, '283.220'), (1, '273.940')] [2023-10-12 22:16:03,787][44958] Updated weights for policy 0, policy_version 53800 (0.0010) [2023-10-12 22:16:04,149][44958] Updated weights for policy 0, policy_version 53810 (0.0008) [2023-10-12 22:16:04,526][44958] Updated weights for policy 0, policy_version 53820 (0.0009) [2023-10-12 22:16:05,165][44959] Updated weights for policy 1, policy_version 54090 (0.0008) [2023-10-12 22:16:05,534][44959] Updated weights for policy 1, policy_version 54100 (0.0011) [2023-10-12 22:16:05,895][44959] Updated weights for policy 1, policy_version 54110 (0.0011) [2023-10-12 22:16:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 110526464. Throughput: 0: 1646.3, 1: 1649.9. Samples: 27633094. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) [2023-10-12 22:16:06,444][43579] Avg episode reward: [(0, '283.240'), (1, '277.800')] [2023-10-12 22:16:08,628][44958] Updated weights for policy 0, policy_version 53830 (0.0007) [2023-10-12 22:16:09,000][44958] Updated weights for policy 0, policy_version 53840 (0.0009) [2023-10-12 22:16:09,376][44958] Updated weights for policy 0, policy_version 53850 (0.0008) [2023-10-12 22:16:10,161][44959] Updated weights for policy 1, policy_version 54120 (0.0008) [2023-10-12 22:16:10,539][44959] Updated weights for policy 1, policy_version 54130 (0.0009) [2023-10-12 22:16:10,904][44959] Updated weights for policy 1, policy_version 54140 (0.0008) [2023-10-12 22:16:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 110592000. Throughput: 0: 1643.6, 1: 1641.7. Samples: 27652080. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) [2023-10-12 22:16:11,444][43579] Avg episode reward: [(0, '284.070'), (1, '280.290')] [2023-10-12 22:16:13,642][44958] Updated weights for policy 0, policy_version 53860 (0.0008) [2023-10-12 22:16:14,031][44958] Updated weights for policy 0, policy_version 53870 (0.0008) [2023-10-12 22:16:14,408][44958] Updated weights for policy 0, policy_version 53880 (0.0007) [2023-10-12 22:16:15,146][44959] Updated weights for policy 1, policy_version 54150 (0.0009) [2023-10-12 22:16:15,522][44959] Updated weights for policy 1, policy_version 54160 (0.0007) [2023-10-12 22:16:15,899][44959] Updated weights for policy 1, policy_version 54170 (0.0009) [2023-10-12 22:16:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 110657536. Throughput: 0: 1643.4, 1: 1639.9. Samples: 27671334. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) [2023-10-12 22:16:16,444][43579] Avg episode reward: [(0, '281.070'), (1, '278.960')] [2023-10-12 22:16:18,540][44958] Updated weights for policy 0, policy_version 53890 (0.0008) [2023-10-12 22:16:18,922][44958] Updated weights for policy 0, policy_version 53900 (0.0009) [2023-10-12 22:16:19,298][44958] Updated weights for policy 0, policy_version 53910 (0.0008) [2023-10-12 22:16:19,669][44958] Updated weights for policy 0, policy_version 53920 (0.0010) [2023-10-12 22:16:20,140][44959] Updated weights for policy 1, policy_version 54180 (0.0009) [2023-10-12 22:16:20,509][44959] Updated weights for policy 1, policy_version 54190 (0.0009) [2023-10-12 22:16:20,880][44959] Updated weights for policy 1, policy_version 54200 (0.0009) [2023-10-12 22:16:21,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 110723072. Throughput: 0: 1644.5, 1: 1642.5. Samples: 27681828. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) [2023-10-12 22:16:21,443][43579] Avg episode reward: [(0, '280.290'), (1, '276.130')] [2023-10-12 22:16:23,764][44958] Updated weights for policy 0, policy_version 53930 (0.0007) [2023-10-12 22:16:24,133][44958] Updated weights for policy 0, policy_version 53940 (0.0007) [2023-10-12 22:16:24,511][44958] Updated weights for policy 0, policy_version 53950 (0.0007) [2023-10-12 22:16:25,245][44959] Updated weights for policy 1, policy_version 54210 (0.0009) [2023-10-12 22:16:25,623][44959] Updated weights for policy 1, policy_version 54220 (0.0007) [2023-10-12 22:16:25,989][44959] Updated weights for policy 1, policy_version 54230 (0.0008) [2023-10-12 22:16:26,351][44959] Updated weights for policy 1, policy_version 54240 (0.0008) [2023-10-12 22:16:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 110788608. Throughput: 0: 1641.0, 1: 1641.8. Samples: 27701370. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) [2023-10-12 22:16:26,444][43579] Avg episode reward: [(0, '277.750'), (1, '274.960')] [2023-10-12 22:16:28,626][44958] Updated weights for policy 0, policy_version 53960 (0.0008) [2023-10-12 22:16:28,991][44958] Updated weights for policy 0, policy_version 53970 (0.0008) [2023-10-12 22:16:29,358][44958] Updated weights for policy 0, policy_version 53980 (0.0008) [2023-10-12 22:16:30,317][44959] Updated weights for policy 1, policy_version 54250 (0.0010) [2023-10-12 22:16:30,672][44959] Updated weights for policy 1, policy_version 54260 (0.0009) [2023-10-12 22:16:31,044][44959] Updated weights for policy 1, policy_version 54270 (0.0011) [2023-10-12 22:16:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 110854144. Throughput: 0: 1638.4, 1: 1641.8. Samples: 27720732. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 22:16:31,443][43579] Avg episode reward: [(0, '275.650'), (1, '279.790')] [2023-10-12 22:16:33,685][44958] Updated weights for policy 0, policy_version 53990 (0.0009) [2023-10-12 22:16:34,058][44958] Updated weights for policy 0, policy_version 54000 (0.0009) [2023-10-12 22:16:34,439][44958] Updated weights for policy 0, policy_version 54010 (0.0009) [2023-10-12 22:16:35,335][44959] Updated weights for policy 1, policy_version 54280 (0.0010) [2023-10-12 22:16:35,701][44959] Updated weights for policy 1, policy_version 54290 (0.0007) [2023-10-12 22:16:36,064][44959] Updated weights for policy 1, policy_version 54300 (0.0009) [2023-10-12 22:16:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 110919680. Throughput: 0: 1641.5, 1: 1641.2. Samples: 27731242. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 22:16:36,443][43579] Avg episode reward: [(0, '274.610'), (1, '275.710')] [2023-10-12 22:16:38,230][44958] Updated weights for policy 0, policy_version 54020 (0.0009) [2023-10-12 22:16:38,605][44958] Updated weights for policy 0, policy_version 54030 (0.0008) [2023-10-12 22:16:38,971][44958] Updated weights for policy 0, policy_version 54040 (0.0009) [2023-10-12 22:16:40,187][44959] Updated weights for policy 1, policy_version 54310 (0.0010) [2023-10-12 22:16:40,560][44959] Updated weights for policy 1, policy_version 54320 (0.0007) [2023-10-12 22:16:40,928][44959] Updated weights for policy 1, policy_version 54330 (0.0009) [2023-10-12 22:16:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 110985216. Throughput: 0: 1643.0, 1: 1642.6. Samples: 27750964. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 22:16:41,444][43579] Avg episode reward: [(0, '279.790'), (1, '275.100')] [2023-10-12 22:16:42,986][44958] Updated weights for policy 0, policy_version 54050 (0.0008) [2023-10-12 22:16:43,371][44958] Updated weights for policy 0, policy_version 54060 (0.0010) [2023-10-12 22:16:43,735][44958] Updated weights for policy 0, policy_version 54070 (0.0009) [2023-10-12 22:16:44,110][44958] Updated weights for policy 0, policy_version 54080 (0.0008) [2023-10-12 22:16:45,017][44959] Updated weights for policy 1, policy_version 54340 (0.0009) [2023-10-12 22:16:45,390][44959] Updated weights for policy 1, policy_version 54350 (0.0008) [2023-10-12 22:16:45,764][44959] Updated weights for policy 1, policy_version 54360 (0.0008) [2023-10-12 22:16:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111050752. Throughput: 0: 1644.8, 1: 1645.9. Samples: 27770582. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 22:16:46,444][43579] Avg episode reward: [(0, '277.050'), (1, '270.670')] [2023-10-12 22:16:48,479][44958] Updated weights for policy 0, policy_version 54090 (0.0009) [2023-10-12 22:16:48,849][44958] Updated weights for policy 0, policy_version 54100 (0.0008) [2023-10-12 22:16:49,220][44958] Updated weights for policy 0, policy_version 54110 (0.0008) [2023-10-12 22:16:50,056][44959] Updated weights for policy 1, policy_version 54370 (0.0008) [2023-10-12 22:16:50,460][44959] Updated weights for policy 1, policy_version 54380 (0.0008) [2023-10-12 22:16:50,817][44959] Updated weights for policy 1, policy_version 54390 (0.0007) [2023-10-12 22:16:51,187][44959] Updated weights for policy 1, policy_version 54400 (0.0008) [2023-10-12 22:16:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111116288. Throughput: 0: 1636.7, 1: 1639.3. Samples: 27780514. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 22:16:51,443][43579] Avg episode reward: [(0, '277.750'), (1, '269.470')] [2023-10-12 22:16:53,421][44958] Updated weights for policy 0, policy_version 54120 (0.0008) [2023-10-12 22:16:53,788][44958] Updated weights for policy 0, policy_version 54130 (0.0007) [2023-10-12 22:16:54,155][44958] Updated weights for policy 0, policy_version 54140 (0.0010) [2023-10-12 22:16:55,274][44959] Updated weights for policy 1, policy_version 54410 (0.0009) [2023-10-12 22:16:55,643][44959] Updated weights for policy 1, policy_version 54420 (0.0010) [2023-10-12 22:16:56,008][44959] Updated weights for policy 1, policy_version 54430 (0.0010) [2023-10-12 22:16:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111181824. Throughput: 0: 1645.9, 1: 1648.9. Samples: 27800344. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 22:16:56,443][43579] Avg episode reward: [(0, '280.090'), (1, '272.540')] [2023-10-12 22:16:58,045][44958] Updated weights for policy 0, policy_version 54150 (0.0009) [2023-10-12 22:16:58,406][44958] Updated weights for policy 0, policy_version 54160 (0.0008) [2023-10-12 22:16:58,784][44958] Updated weights for policy 0, policy_version 54170 (0.0007) [2023-10-12 22:16:59,998][44959] Updated weights for policy 1, policy_version 54440 (0.0009) [2023-10-12 22:17:00,365][44959] Updated weights for policy 1, policy_version 54450 (0.0011) [2023-10-12 22:17:00,736][44959] Updated weights for policy 1, policy_version 54460 (0.0010) [2023-10-12 22:17:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111247360. Throughput: 0: 1652.4, 1: 1649.6. Samples: 27819926. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-10-12 22:17:01,444][43579] Avg episode reward: [(0, '276.920'), (1, '275.820')] [2023-10-12 22:17:03,183][44958] Updated weights for policy 0, policy_version 54180 (0.0010) [2023-10-12 22:17:03,564][44958] Updated weights for policy 0, policy_version 54190 (0.0010) [2023-10-12 22:17:03,944][44958] Updated weights for policy 0, policy_version 54200 (0.0010) [2023-10-12 22:17:04,907][44959] Updated weights for policy 1, policy_version 54470 (0.0009) [2023-10-12 22:17:05,272][44959] Updated weights for policy 1, policy_version 54480 (0.0010) [2023-10-12 22:17:05,645][44959] Updated weights for policy 1, policy_version 54490 (0.0008) [2023-10-12 22:17:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111312896. Throughput: 0: 1640.8, 1: 1651.9. Samples: 27830000. Policy #0 lag: (min: 27.0, avg: 40.9, max: 59.0) [2023-10-12 22:17:06,444][43579] Avg episode reward: [(0, '274.330'), (1, '274.040')] [2023-10-12 22:17:08,308][44958] Updated weights for policy 0, policy_version 54210 (0.0009) [2023-10-12 22:17:08,678][44958] Updated weights for policy 0, policy_version 54220 (0.0007) [2023-10-12 22:17:09,044][44958] Updated weights for policy 0, policy_version 54230 (0.0008) [2023-10-12 22:17:09,424][44958] Updated weights for policy 0, policy_version 54240 (0.0007) [2023-10-12 22:17:09,983][44959] Updated weights for policy 1, policy_version 54500 (0.0011) [2023-10-12 22:17:10,344][44959] Updated weights for policy 1, policy_version 54510 (0.0008) [2023-10-12 22:17:10,714][44959] Updated weights for policy 1, policy_version 54520 (0.0009) [2023-10-12 22:17:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111378432. Throughput: 0: 1650.6, 1: 1649.1. Samples: 27849854. Policy #0 lag: (min: 27.0, avg: 40.9, max: 59.0) [2023-10-12 22:17:11,444][43579] Avg episode reward: [(0, '273.200'), (1, '275.050')] [2023-10-12 22:17:13,475][44958] Updated weights for policy 0, policy_version 54250 (0.0007) [2023-10-12 22:17:13,844][44958] Updated weights for policy 0, policy_version 54260 (0.0008) [2023-10-12 22:17:14,205][44958] Updated weights for policy 0, policy_version 54270 (0.0007) [2023-10-12 22:17:14,678][44959] Updated weights for policy 1, policy_version 54530 (0.0007) [2023-10-12 22:17:15,041][44959] Updated weights for policy 1, policy_version 54540 (0.0008) [2023-10-12 22:17:15,411][44959] Updated weights for policy 1, policy_version 54550 (0.0009) [2023-10-12 22:17:15,776][44959] Updated weights for policy 1, policy_version 54560 (0.0010) [2023-10-12 22:17:16,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111443968. Throughput: 0: 1652.4, 1: 1655.7. Samples: 27869598. Policy #0 lag: (min: 27.0, avg: 40.9, max: 59.0) [2023-10-12 22:17:16,443][43579] Avg episode reward: [(0, '278.010'), (1, '274.450')] [2023-10-12 22:17:18,371][44958] Updated weights for policy 0, policy_version 54280 (0.0007) [2023-10-12 22:17:18,739][44958] Updated weights for policy 0, policy_version 54290 (0.0008) [2023-10-12 22:17:19,118][44958] Updated weights for policy 0, policy_version 54300 (0.0009) [2023-10-12 22:17:19,959][44959] Updated weights for policy 1, policy_version 54570 (0.0008) [2023-10-12 22:17:20,334][44959] Updated weights for policy 1, policy_version 54580 (0.0009) [2023-10-12 22:17:20,707][44959] Updated weights for policy 1, policy_version 54590 (0.0008) [2023-10-12 22:17:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111509504. Throughput: 0: 1639.8, 1: 1658.0. Samples: 27879644. Policy #0 lag: (min: 27.0, avg: 40.9, max: 59.0) [2023-10-12 22:17:21,443][43579] Avg episode reward: [(0, '277.410'), (1, '276.790')] [2023-10-12 22:17:23,280][44958] Updated weights for policy 0, policy_version 54310 (0.0007) [2023-10-12 22:17:23,658][44958] Updated weights for policy 0, policy_version 54320 (0.0008) [2023-10-12 22:17:24,032][44958] Updated weights for policy 0, policy_version 54330 (0.0011) [2023-10-12 22:17:24,774][44959] Updated weights for policy 1, policy_version 54600 (0.0007) [2023-10-12 22:17:25,146][44959] Updated weights for policy 1, policy_version 54610 (0.0007) [2023-10-12 22:17:25,525][44959] Updated weights for policy 1, policy_version 54620 (0.0008) [2023-10-12 22:17:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111575040. Throughput: 0: 1646.0, 1: 1651.8. Samples: 27899362. Policy #0 lag: (min: 27.0, avg: 40.9, max: 59.0) [2023-10-12 22:17:26,443][43579] Avg episode reward: [(0, '273.140'), (1, '277.060')] [2023-10-12 22:17:28,309][44958] Updated weights for policy 0, policy_version 54340 (0.0009) [2023-10-12 22:17:28,680][44958] Updated weights for policy 0, policy_version 54350 (0.0009) [2023-10-12 22:17:29,056][44958] Updated weights for policy 0, policy_version 54360 (0.0009) [2023-10-12 22:17:29,638][44959] Updated weights for policy 1, policy_version 54630 (0.0008) [2023-10-12 22:17:30,016][44959] Updated weights for policy 1, policy_version 54640 (0.0007) [2023-10-12 22:17:30,382][44959] Updated weights for policy 1, policy_version 54650 (0.0007) [2023-10-12 22:17:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111640576. Throughput: 0: 1638.8, 1: 1660.5. Samples: 27919046. Policy #0 lag: (min: 27.0, avg: 40.9, max: 59.0) [2023-10-12 22:17:31,443][43579] Avg episode reward: [(0, '278.150'), (1, '281.470')] [2023-10-12 22:17:33,270][44958] Updated weights for policy 0, policy_version 54370 (0.0009) [2023-10-12 22:17:33,638][44958] Updated weights for policy 0, policy_version 54380 (0.0008) [2023-10-12 22:17:34,004][44958] Updated weights for policy 0, policy_version 54390 (0.0007) [2023-10-12 22:17:34,378][44958] Updated weights for policy 0, policy_version 54400 (0.0007) [2023-10-12 22:17:34,476][44959] Updated weights for policy 1, policy_version 54660 (0.0009) [2023-10-12 22:17:34,834][44959] Updated weights for policy 1, policy_version 54670 (0.0008) [2023-10-12 22:17:35,198][44959] Updated weights for policy 1, policy_version 54680 (0.0010) [2023-10-12 22:17:36,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 111706112. Throughput: 0: 1642.7, 1: 1661.7. Samples: 27929214. Policy #0 lag: (min: 27.0, avg: 40.9, max: 59.0) [2023-10-12 22:17:36,444][43579] Avg episode reward: [(0, '281.950'), (1, '279.010')] [2023-10-12 22:17:38,428][44958] Updated weights for policy 0, policy_version 54410 (0.0009) [2023-10-12 22:17:38,814][44958] Updated weights for policy 0, policy_version 54420 (0.0007) [2023-10-12 22:17:39,187][44958] Updated weights for policy 0, policy_version 54430 (0.0007) [2023-10-12 22:17:39,468][44959] Updated weights for policy 1, policy_version 54690 (0.0009) [2023-10-12 22:17:39,848][44959] Updated weights for policy 1, policy_version 54700 (0.0011) [2023-10-12 22:17:40,217][44959] Updated weights for policy 1, policy_version 54710 (0.0008) [2023-10-12 22:17:40,585][44959] Updated weights for policy 1, policy_version 54720 (0.0009) [2023-10-12 22:17:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111771648. Throughput: 0: 1646.4, 1: 1651.2. Samples: 27948736. Policy #0 lag: (min: 25.0, avg: 41.5, max: 57.0) [2023-10-12 22:17:41,443][43579] Avg episode reward: [(0, '279.000'), (1, '279.290')] [2023-10-12 22:17:43,313][44958] Updated weights for policy 0, policy_version 54440 (0.0009) [2023-10-12 22:17:43,690][44958] Updated weights for policy 0, policy_version 54450 (0.0008) [2023-10-12 22:17:44,054][44958] Updated weights for policy 0, policy_version 54460 (0.0009) [2023-10-12 22:17:44,895][44959] Updated weights for policy 1, policy_version 54730 (0.0007) [2023-10-12 22:17:45,254][44959] Updated weights for policy 1, policy_version 54740 (0.0010) [2023-10-12 22:17:45,628][44959] Updated weights for policy 1, policy_version 54750 (0.0010) [2023-10-12 22:17:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111837184. Throughput: 0: 1641.5, 1: 1651.2. Samples: 27968100. Policy #0 lag: (min: 25.0, avg: 41.5, max: 57.0) [2023-10-12 22:17:46,444][43579] Avg episode reward: [(0, '279.880'), (1, '284.980')] [2023-10-12 22:17:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000054464_55771136.pth... [2023-10-12 22:17:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000054752_56066048.pth... [2023-10-12 22:17:46,483][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000053216_54493184.pth [2023-10-12 22:17:46,494][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000052928_54198272.pth [2023-10-12 22:17:48,276][44958] Updated weights for policy 0, policy_version 54470 (0.0009) [2023-10-12 22:17:48,655][44958] Updated weights for policy 0, policy_version 54480 (0.0008) [2023-10-12 22:17:49,033][44958] Updated weights for policy 0, policy_version 54490 (0.0008) [2023-10-12 22:17:49,778][44959] Updated weights for policy 1, policy_version 54760 (0.0007) [2023-10-12 22:17:50,140][44959] Updated weights for policy 1, policy_version 54770 (0.0007) [2023-10-12 22:17:50,511][44959] Updated weights for policy 1, policy_version 54780 (0.0007) [2023-10-12 22:17:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111902720. Throughput: 0: 1644.3, 1: 1646.9. Samples: 27978104. Policy #0 lag: (min: 25.0, avg: 41.5, max: 57.0) [2023-10-12 22:17:51,444][43579] Avg episode reward: [(0, '276.380'), (1, '285.400')] [2023-10-12 22:17:53,193][44958] Updated weights for policy 0, policy_version 54500 (0.0009) [2023-10-12 22:17:53,563][44958] Updated weights for policy 0, policy_version 54510 (0.0008) [2023-10-12 22:17:53,928][44958] Updated weights for policy 0, policy_version 54520 (0.0009) [2023-10-12 22:17:54,659][44959] Updated weights for policy 1, policy_version 54790 (0.0008) [2023-10-12 22:17:55,038][44959] Updated weights for policy 1, policy_version 54800 (0.0008) [2023-10-12 22:17:55,405][44959] Updated weights for policy 1, policy_version 54810 (0.0009) [2023-10-12 22:17:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 111968256. Throughput: 0: 1647.4, 1: 1639.2. Samples: 27997750. Policy #0 lag: (min: 25.0, avg: 41.5, max: 57.0) [2023-10-12 22:17:56,444][43579] Avg episode reward: [(0, '278.270'), (1, '278.950')] [2023-10-12 22:17:58,010][44958] Updated weights for policy 0, policy_version 54530 (0.0010) [2023-10-12 22:17:58,368][44958] Updated weights for policy 0, policy_version 54540 (0.0008) [2023-10-12 22:17:58,748][44958] Updated weights for policy 0, policy_version 54550 (0.0008) [2023-10-12 22:17:59,119][44958] Updated weights for policy 0, policy_version 54560 (0.0009) [2023-10-12 22:17:59,732][44959] Updated weights for policy 1, policy_version 54820 (0.0009) [2023-10-12 22:18:00,093][44959] Updated weights for policy 1, policy_version 54830 (0.0011) [2023-10-12 22:18:00,455][44959] Updated weights for policy 1, policy_version 54840 (0.0008) [2023-10-12 22:18:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 112033792. Throughput: 0: 1642.5, 1: 1637.0. Samples: 28017178. Policy #0 lag: (min: 25.0, avg: 41.5, max: 57.0) [2023-10-12 22:18:01,443][43579] Avg episode reward: [(0, '273.770'), (1, '279.020')] [2023-10-12 22:18:03,295][44958] Updated weights for policy 0, policy_version 54570 (0.0009) [2023-10-12 22:18:03,676][44958] Updated weights for policy 0, policy_version 54580 (0.0007) [2023-10-12 22:18:04,053][44958] Updated weights for policy 0, policy_version 54590 (0.0007) [2023-10-12 22:18:04,530][44959] Updated weights for policy 1, policy_version 54850 (0.0007) [2023-10-12 22:18:04,904][44959] Updated weights for policy 1, policy_version 54860 (0.0007) [2023-10-12 22:18:05,280][44959] Updated weights for policy 1, policy_version 54870 (0.0007) [2023-10-12 22:18:05,640][44959] Updated weights for policy 1, policy_version 54880 (0.0008) [2023-10-12 22:18:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112099328. Throughput: 0: 1643.8, 1: 1639.7. Samples: 28027402. Policy #0 lag: (min: 25.0, avg: 41.5, max: 57.0) [2023-10-12 22:18:06,443][43579] Avg episode reward: [(0, '274.020'), (1, '277.400')] [2023-10-12 22:18:08,060][44958] Updated weights for policy 0, policy_version 54600 (0.0009) [2023-10-12 22:18:08,436][44958] Updated weights for policy 0, policy_version 54610 (0.0009) [2023-10-12 22:18:08,813][44958] Updated weights for policy 0, policy_version 54620 (0.0009) [2023-10-12 22:18:09,863][44959] Updated weights for policy 1, policy_version 54890 (0.0007) [2023-10-12 22:18:10,233][44959] Updated weights for policy 1, policy_version 54900 (0.0007) [2023-10-12 22:18:10,610][44959] Updated weights for policy 1, policy_version 54910 (0.0009) [2023-10-12 22:18:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112164864. Throughput: 0: 1648.6, 1: 1638.1. Samples: 28047266. Policy #0 lag: (min: 25.0, avg: 41.5, max: 57.0) [2023-10-12 22:18:11,444][43579] Avg episode reward: [(0, '278.780'), (1, '278.490')] [2023-10-12 22:18:13,026][44958] Updated weights for policy 0, policy_version 54630 (0.0009) [2023-10-12 22:18:13,402][44958] Updated weights for policy 0, policy_version 54640 (0.0010) [2023-10-12 22:18:13,780][44958] Updated weights for policy 0, policy_version 54650 (0.0009) [2023-10-12 22:18:14,656][44959] Updated weights for policy 1, policy_version 54920 (0.0009) [2023-10-12 22:18:15,022][44959] Updated weights for policy 1, policy_version 54930 (0.0007) [2023-10-12 22:18:15,395][44959] Updated weights for policy 1, policy_version 54940 (0.0007) [2023-10-12 22:18:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112230400. Throughput: 0: 1651.9, 1: 1627.1. Samples: 28066600. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-12 22:18:16,444][43579] Avg episode reward: [(0, '280.600'), (1, '279.960')] [2023-10-12 22:18:17,953][44958] Updated weights for policy 0, policy_version 54660 (0.0009) [2023-10-12 22:18:18,327][44958] Updated weights for policy 0, policy_version 54670 (0.0008) [2023-10-12 22:18:18,697][44958] Updated weights for policy 0, policy_version 54680 (0.0009) [2023-10-12 22:18:19,661][44959] Updated weights for policy 1, policy_version 54950 (0.0008) [2023-10-12 22:18:20,031][44959] Updated weights for policy 1, policy_version 54960 (0.0008) [2023-10-12 22:18:20,403][44959] Updated weights for policy 1, policy_version 54970 (0.0008) [2023-10-12 22:18:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112295936. Throughput: 0: 1642.1, 1: 1634.6. Samples: 28076666. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-12 22:18:21,443][43579] Avg episode reward: [(0, '280.290'), (1, '275.130')] [2023-10-12 22:18:22,848][44958] Updated weights for policy 0, policy_version 54690 (0.0008) [2023-10-12 22:18:23,221][44958] Updated weights for policy 0, policy_version 54700 (0.0008) [2023-10-12 22:18:23,585][44958] Updated weights for policy 0, policy_version 54710 (0.0009) [2023-10-12 22:18:23,966][44958] Updated weights for policy 0, policy_version 54720 (0.0008) [2023-10-12 22:18:24,545][44959] Updated weights for policy 1, policy_version 54980 (0.0009) [2023-10-12 22:18:24,934][44959] Updated weights for policy 1, policy_version 54990 (0.0008) [2023-10-12 22:18:25,299][44959] Updated weights for policy 1, policy_version 55000 (0.0009) [2023-10-12 22:18:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112361472. Throughput: 0: 1649.3, 1: 1634.4. Samples: 28096502. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-12 22:18:26,444][43579] Avg episode reward: [(0, '279.140'), (1, '277.820')] [2023-10-12 22:18:28,018][44958] Updated weights for policy 0, policy_version 54730 (0.0009) [2023-10-12 22:18:28,393][44958] Updated weights for policy 0, policy_version 54740 (0.0009) [2023-10-12 22:18:28,761][44958] Updated weights for policy 0, policy_version 54750 (0.0009) [2023-10-12 22:18:29,381][44959] Updated weights for policy 1, policy_version 55010 (0.0007) [2023-10-12 22:18:29,744][44959] Updated weights for policy 1, policy_version 55020 (0.0008) [2023-10-12 22:18:30,119][44959] Updated weights for policy 1, policy_version 55030 (0.0007) [2023-10-12 22:18:30,485][44959] Updated weights for policy 1, policy_version 55040 (0.0008) [2023-10-12 22:18:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112427008. Throughput: 0: 1650.1, 1: 1643.0. Samples: 28116290. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-12 22:18:31,443][43579] Avg episode reward: [(0, '284.220'), (1, '273.230')] [2023-10-12 22:18:32,993][44958] Updated weights for policy 0, policy_version 54760 (0.0008) [2023-10-12 22:18:33,377][44958] Updated weights for policy 0, policy_version 54770 (0.0008) [2023-10-12 22:18:33,744][44958] Updated weights for policy 0, policy_version 54780 (0.0010) [2023-10-12 22:18:34,747][44959] Updated weights for policy 1, policy_version 55050 (0.0008) [2023-10-12 22:18:35,123][44959] Updated weights for policy 1, policy_version 55060 (0.0009) [2023-10-12 22:18:35,485][44959] Updated weights for policy 1, policy_version 55070 (0.0010) [2023-10-12 22:18:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 112492544. Throughput: 0: 1646.5, 1: 1649.1. Samples: 28126408. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-12 22:18:36,443][43579] Avg episode reward: [(0, '280.550'), (1, '277.380')] [2023-10-12 22:18:37,957][44958] Updated weights for policy 0, policy_version 54790 (0.0008) [2023-10-12 22:18:38,325][44958] Updated weights for policy 0, policy_version 54800 (0.0007) [2023-10-12 22:18:38,698][44958] Updated weights for policy 0, policy_version 54810 (0.0008) [2023-10-12 22:18:39,683][44959] Updated weights for policy 1, policy_version 55080 (0.0008) [2023-10-12 22:18:40,057][44959] Updated weights for policy 1, policy_version 55090 (0.0008) [2023-10-12 22:18:40,425][44959] Updated weights for policy 1, policy_version 55100 (0.0007) [2023-10-12 22:18:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112558080. Throughput: 0: 1651.3, 1: 1648.3. Samples: 28146228. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-12 22:18:41,443][43579] Avg episode reward: [(0, '277.450'), (1, '278.060')] [2023-10-12 22:18:42,804][44958] Updated weights for policy 0, policy_version 54820 (0.0008) [2023-10-12 22:18:43,172][44958] Updated weights for policy 0, policy_version 54830 (0.0007) [2023-10-12 22:18:43,549][44958] Updated weights for policy 0, policy_version 54840 (0.0008) [2023-10-12 22:18:44,369][44959] Updated weights for policy 1, policy_version 55110 (0.0009) [2023-10-12 22:18:44,737][44959] Updated weights for policy 1, policy_version 55120 (0.0008) [2023-10-12 22:18:45,110][44959] Updated weights for policy 1, policy_version 55130 (0.0007) [2023-10-12 22:18:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 112623616. Throughput: 0: 1651.5, 1: 1655.9. Samples: 28166012. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-12 22:18:46,443][43579] Avg episode reward: [(0, '282.450'), (1, '279.840')] [2023-10-12 22:18:47,680][44958] Updated weights for policy 0, policy_version 54850 (0.0008) [2023-10-12 22:18:48,057][44958] Updated weights for policy 0, policy_version 54860 (0.0008) [2023-10-12 22:18:48,435][44958] Updated weights for policy 0, policy_version 54870 (0.0010) [2023-10-12 22:18:48,798][44958] Updated weights for policy 0, policy_version 54880 (0.0011) [2023-10-12 22:18:49,225][44959] Updated weights for policy 1, policy_version 55140 (0.0010) [2023-10-12 22:18:49,594][44959] Updated weights for policy 1, policy_version 55150 (0.0009) [2023-10-12 22:18:49,967][44959] Updated weights for policy 1, policy_version 55160 (0.0010) [2023-10-12 22:18:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112689152. Throughput: 0: 1649.7, 1: 1655.4. Samples: 28176134. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) [2023-10-12 22:18:51,444][43579] Avg episode reward: [(0, '283.660'), (1, '279.650')] [2023-10-12 22:18:52,854][44958] Updated weights for policy 0, policy_version 54890 (0.0007) [2023-10-12 22:18:53,233][44958] Updated weights for policy 0, policy_version 54900 (0.0008) [2023-10-12 22:18:53,598][44958] Updated weights for policy 0, policy_version 54910 (0.0009) [2023-10-12 22:18:54,196][44959] Updated weights for policy 1, policy_version 55170 (0.0009) [2023-10-12 22:18:54,573][44959] Updated weights for policy 1, policy_version 55180 (0.0009) [2023-10-12 22:18:54,933][44959] Updated weights for policy 1, policy_version 55190 (0.0008) [2023-10-12 22:18:55,301][44959] Updated weights for policy 1, policy_version 55200 (0.0009) [2023-10-12 22:18:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112754688. Throughput: 0: 1654.3, 1: 1639.7. Samples: 28195496. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) [2023-10-12 22:18:56,443][43579] Avg episode reward: [(0, '282.510'), (1, '282.320')] [2023-10-12 22:18:57,771][44958] Updated weights for policy 0, policy_version 54920 (0.0007) [2023-10-12 22:18:58,150][44958] Updated weights for policy 0, policy_version 54930 (0.0008) [2023-10-12 22:18:58,518][44958] Updated weights for policy 0, policy_version 54940 (0.0010) [2023-10-12 22:18:59,360][44959] Updated weights for policy 1, policy_version 55210 (0.0009) [2023-10-12 22:18:59,719][44959] Updated weights for policy 1, policy_version 55220 (0.0007) [2023-10-12 22:19:00,091][44959] Updated weights for policy 1, policy_version 55230 (0.0007) [2023-10-12 22:19:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112820224. Throughput: 0: 1653.3, 1: 1657.3. Samples: 28215578. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) [2023-10-12 22:19:01,443][43579] Avg episode reward: [(0, '277.200'), (1, '280.380')] [2023-10-12 22:19:02,784][44958] Updated weights for policy 0, policy_version 54950 (0.0010) [2023-10-12 22:19:03,150][44958] Updated weights for policy 0, policy_version 54960 (0.0010) [2023-10-12 22:19:03,520][44958] Updated weights for policy 0, policy_version 54970 (0.0007) [2023-10-12 22:19:04,276][44959] Updated weights for policy 1, policy_version 55240 (0.0008) [2023-10-12 22:19:04,645][44959] Updated weights for policy 1, policy_version 55250 (0.0008) [2023-10-12 22:19:05,020][44959] Updated weights for policy 1, policy_version 55260 (0.0008) [2023-10-12 22:19:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112885760. Throughput: 0: 1651.5, 1: 1653.9. Samples: 28225408. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) [2023-10-12 22:19:06,444][43579] Avg episode reward: [(0, '276.080'), (1, '278.540')] [2023-10-12 22:19:07,751][44958] Updated weights for policy 0, policy_version 54980 (0.0010) [2023-10-12 22:19:08,120][44958] Updated weights for policy 0, policy_version 54990 (0.0011) [2023-10-12 22:19:08,492][44958] Updated weights for policy 0, policy_version 55000 (0.0011) [2023-10-12 22:19:09,191][44959] Updated weights for policy 1, policy_version 55270 (0.0007) [2023-10-12 22:19:09,547][44959] Updated weights for policy 1, policy_version 55280 (0.0008) [2023-10-12 22:19:09,911][44959] Updated weights for policy 1, policy_version 55290 (0.0007) [2023-10-12 22:19:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 112951296. Throughput: 0: 1646.5, 1: 1645.5. Samples: 28244640. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) [2023-10-12 22:19:11,443][43579] Avg episode reward: [(0, '277.940'), (1, '276.530')] [2023-10-12 22:19:12,450][44958] Updated weights for policy 0, policy_version 55010 (0.0009) [2023-10-12 22:19:12,822][44958] Updated weights for policy 0, policy_version 55020 (0.0008) [2023-10-12 22:19:13,196][44958] Updated weights for policy 0, policy_version 55030 (0.0008) [2023-10-12 22:19:13,565][44958] Updated weights for policy 0, policy_version 55040 (0.0009) [2023-10-12 22:19:14,100][44959] Updated weights for policy 1, policy_version 55300 (0.0010) [2023-10-12 22:19:14,505][44959] Updated weights for policy 1, policy_version 55310 (0.0010) [2023-10-12 22:19:14,877][44959] Updated weights for policy 1, policy_version 55320 (0.0008) [2023-10-12 22:19:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113016832. Throughput: 0: 1649.3, 1: 1654.9. Samples: 28264980. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) [2023-10-12 22:19:16,443][43579] Avg episode reward: [(0, '272.600'), (1, '278.850')] [2023-10-12 22:19:17,826][44958] Updated weights for policy 0, policy_version 55050 (0.0009) [2023-10-12 22:19:18,199][44958] Updated weights for policy 0, policy_version 55060 (0.0008) [2023-10-12 22:19:18,578][44958] Updated weights for policy 0, policy_version 55070 (0.0009) [2023-10-12 22:19:18,909][44959] Updated weights for policy 1, policy_version 55330 (0.0011) [2023-10-12 22:19:19,273][44959] Updated weights for policy 1, policy_version 55340 (0.0009) [2023-10-12 22:19:19,635][44959] Updated weights for policy 1, policy_version 55350 (0.0008) [2023-10-12 22:19:20,010][44959] Updated weights for policy 1, policy_version 55360 (0.0008) [2023-10-12 22:19:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113082368. Throughput: 0: 1652.6, 1: 1644.8. Samples: 28274792. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) [2023-10-12 22:19:21,443][43579] Avg episode reward: [(0, '273.420'), (1, '281.770')] [2023-10-12 22:19:22,450][44958] Updated weights for policy 0, policy_version 55080 (0.0010) [2023-10-12 22:19:22,828][44958] Updated weights for policy 0, policy_version 55090 (0.0009) [2023-10-12 22:19:23,195][44958] Updated weights for policy 0, policy_version 55100 (0.0010) [2023-10-12 22:19:24,051][44959] Updated weights for policy 1, policy_version 55370 (0.0008) [2023-10-12 22:19:24,417][44959] Updated weights for policy 1, policy_version 55380 (0.0010) [2023-10-12 22:19:24,791][44959] Updated weights for policy 1, policy_version 55390 (0.0008) [2023-10-12 22:19:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113147904. Throughput: 0: 1653.2, 1: 1637.4. Samples: 28294304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:19:26,444][43579] Avg episode reward: [(0, '277.570'), (1, '280.320')] [2023-10-12 22:19:27,454][44958] Updated weights for policy 0, policy_version 55110 (0.0007) [2023-10-12 22:19:27,830][44958] Updated weights for policy 0, policy_version 55120 (0.0007) [2023-10-12 22:19:28,199][44958] Updated weights for policy 0, policy_version 55130 (0.0007) [2023-10-12 22:19:29,050][44959] Updated weights for policy 1, policy_version 55400 (0.0010) [2023-10-12 22:19:29,427][44959] Updated weights for policy 1, policy_version 55410 (0.0010) [2023-10-12 22:19:29,799][44959] Updated weights for policy 1, policy_version 55420 (0.0009) [2023-10-12 22:19:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113213440. Throughput: 0: 1649.4, 1: 1648.7. Samples: 28314426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:19:31,443][43579] Avg episode reward: [(0, '280.290'), (1, '281.230')] [2023-10-12 22:19:32,327][44958] Updated weights for policy 0, policy_version 55140 (0.0009) [2023-10-12 22:19:32,693][44958] Updated weights for policy 0, policy_version 55150 (0.0007) [2023-10-12 22:19:33,072][44958] Updated weights for policy 0, policy_version 55160 (0.0007) [2023-10-12 22:19:34,115][44959] Updated weights for policy 1, policy_version 55430 (0.0008) [2023-10-12 22:19:34,481][44959] Updated weights for policy 1, policy_version 55440 (0.0009) [2023-10-12 22:19:34,865][44959] Updated weights for policy 1, policy_version 55450 (0.0010) [2023-10-12 22:19:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113278976. Throughput: 0: 1650.4, 1: 1639.5. Samples: 28324182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:19:36,444][43579] Avg episode reward: [(0, '279.590'), (1, '280.100')] [2023-10-12 22:19:37,086][44958] Updated weights for policy 0, policy_version 55170 (0.0008) [2023-10-12 22:19:37,456][44958] Updated weights for policy 0, policy_version 55180 (0.0009) [2023-10-12 22:19:37,833][44958] Updated weights for policy 0, policy_version 55190 (0.0009) [2023-10-12 22:19:38,200][44958] Updated weights for policy 0, policy_version 55200 (0.0008) [2023-10-12 22:19:38,877][44959] Updated weights for policy 1, policy_version 55460 (0.0009) [2023-10-12 22:19:39,251][44959] Updated weights for policy 1, policy_version 55470 (0.0008) [2023-10-12 22:19:39,611][44959] Updated weights for policy 1, policy_version 55480 (0.0008) [2023-10-12 22:19:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113344512. Throughput: 0: 1649.2, 1: 1643.6. Samples: 28343676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:19:41,443][43579] Avg episode reward: [(0, '281.500'), (1, '281.600')] [2023-10-12 22:19:42,394][44958] Updated weights for policy 0, policy_version 55210 (0.0007) [2023-10-12 22:19:42,760][44958] Updated weights for policy 0, policy_version 55220 (0.0007) [2023-10-12 22:19:43,138][44958] Updated weights for policy 0, policy_version 55230 (0.0007) [2023-10-12 22:19:43,595][44959] Updated weights for policy 1, policy_version 55490 (0.0009) [2023-10-12 22:19:43,965][44959] Updated weights for policy 1, policy_version 55500 (0.0008) [2023-10-12 22:19:44,330][44959] Updated weights for policy 1, policy_version 55510 (0.0007) [2023-10-12 22:19:44,701][44959] Updated weights for policy 1, policy_version 55520 (0.0007) [2023-10-12 22:19:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113410048. Throughput: 0: 1649.0, 1: 1651.4. Samples: 28364098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:19:46,444][43579] Avg episode reward: [(0, '284.060'), (1, '276.910')] [2023-10-12 22:19:46,456][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000055232_56557568.pth... [2023-10-12 22:19:46,456][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000055520_56852480.pth... [2023-10-12 22:19:46,487][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000053696_54984704.pth [2023-10-12 22:19:46,496][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000053984_55279616.pth [2023-10-12 22:19:47,364][44958] Updated weights for policy 0, policy_version 55240 (0.0008) [2023-10-12 22:19:47,731][44958] Updated weights for policy 0, policy_version 55250 (0.0007) [2023-10-12 22:19:48,095][44958] Updated weights for policy 0, policy_version 55260 (0.0009) [2023-10-12 22:19:49,013][44959] Updated weights for policy 1, policy_version 55530 (0.0009) [2023-10-12 22:19:49,384][44959] Updated weights for policy 1, policy_version 55540 (0.0008) [2023-10-12 22:19:49,749][44959] Updated weights for policy 1, policy_version 55550 (0.0009) [2023-10-12 22:19:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113475584. Throughput: 0: 1653.3, 1: 1641.2. Samples: 28373658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:19:51,443][43579] Avg episode reward: [(0, '276.870'), (1, '277.880')] [2023-10-12 22:19:52,297][44958] Updated weights for policy 0, policy_version 55270 (0.0010) [2023-10-12 22:19:52,666][44958] Updated weights for policy 0, policy_version 55280 (0.0009) [2023-10-12 22:19:53,043][44958] Updated weights for policy 0, policy_version 55290 (0.0007) [2023-10-12 22:19:53,825][44959] Updated weights for policy 1, policy_version 55560 (0.0011) [2023-10-12 22:19:54,187][44959] Updated weights for policy 1, policy_version 55570 (0.0011) [2023-10-12 22:19:54,562][44959] Updated weights for policy 1, policy_version 55580 (0.0010) [2023-10-12 22:19:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113541120. Throughput: 0: 1655.1, 1: 1645.9. Samples: 28393186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:19:56,444][43579] Avg episode reward: [(0, '282.110'), (1, '277.010')] [2023-10-12 22:19:57,289][44958] Updated weights for policy 0, policy_version 55300 (0.0007) [2023-10-12 22:19:57,662][44958] Updated weights for policy 0, policy_version 55310 (0.0008) [2023-10-12 22:19:58,028][44958] Updated weights for policy 0, policy_version 55320 (0.0009) [2023-10-12 22:19:58,603][44959] Updated weights for policy 1, policy_version 55590 (0.0009) [2023-10-12 22:19:58,962][44959] Updated weights for policy 1, policy_version 55600 (0.0009) [2023-10-12 22:19:59,339][44959] Updated weights for policy 1, policy_version 55610 (0.0008) [2023-10-12 22:20:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113606656. Throughput: 0: 1649.0, 1: 1654.2. Samples: 28413624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:20:01,443][43579] Avg episode reward: [(0, '282.360'), (1, '273.740')] [2023-10-12 22:20:02,192][44958] Updated weights for policy 0, policy_version 55330 (0.0010) [2023-10-12 22:20:02,574][44958] Updated weights for policy 0, policy_version 55340 (0.0010) [2023-10-12 22:20:02,944][44958] Updated weights for policy 0, policy_version 55350 (0.0011) [2023-10-12 22:20:03,319][44958] Updated weights for policy 0, policy_version 55360 (0.0008) [2023-10-12 22:20:03,809][44959] Updated weights for policy 1, policy_version 55620 (0.0009) [2023-10-12 22:20:04,205][44959] Updated weights for policy 1, policy_version 55630 (0.0009) [2023-10-12 22:20:04,569][44959] Updated weights for policy 1, policy_version 55640 (0.0010) [2023-10-12 22:20:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 113672192. Throughput: 0: 1649.4, 1: 1647.3. Samples: 28423142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:20:06,443][43579] Avg episode reward: [(0, '278.900'), (1, '271.610')] [2023-10-12 22:20:07,542][44958] Updated weights for policy 0, policy_version 55370 (0.0007) [2023-10-12 22:20:07,923][44958] Updated weights for policy 0, policy_version 55380 (0.0007) [2023-10-12 22:20:08,282][44958] Updated weights for policy 0, policy_version 55390 (0.0008) [2023-10-12 22:20:08,559][44959] Updated weights for policy 1, policy_version 55650 (0.0009) [2023-10-12 22:20:08,930][44959] Updated weights for policy 1, policy_version 55660 (0.0009) [2023-10-12 22:20:09,294][44959] Updated weights for policy 1, policy_version 55670 (0.0007) [2023-10-12 22:20:09,659][44959] Updated weights for policy 1, policy_version 55680 (0.0008) [2023-10-12 22:20:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113737728. Throughput: 0: 1644.7, 1: 1651.5. Samples: 28442630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:20:11,443][43579] Avg episode reward: [(0, '273.670'), (1, '271.990')] [2023-10-12 22:20:12,480][44958] Updated weights for policy 0, policy_version 55400 (0.0008) [2023-10-12 22:20:12,843][44958] Updated weights for policy 0, policy_version 55410 (0.0009) [2023-10-12 22:20:13,218][44958] Updated weights for policy 0, policy_version 55420 (0.0007) [2023-10-12 22:20:13,885][44959] Updated weights for policy 1, policy_version 55690 (0.0011) [2023-10-12 22:20:14,252][44959] Updated weights for policy 1, policy_version 55700 (0.0007) [2023-10-12 22:20:14,630][44959] Updated weights for policy 1, policy_version 55710 (0.0007) [2023-10-12 22:20:16,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113803264. Throughput: 0: 1645.4, 1: 1647.2. Samples: 28462594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:20:16,443][43579] Avg episode reward: [(0, '274.220'), (1, '276.950')] [2023-10-12 22:20:17,657][44958] Updated weights for policy 0, policy_version 55430 (0.0009) [2023-10-12 22:20:18,044][44958] Updated weights for policy 0, policy_version 55440 (0.0011) [2023-10-12 22:20:18,414][44958] Updated weights for policy 0, policy_version 55450 (0.0010) [2023-10-12 22:20:18,786][44959] Updated weights for policy 1, policy_version 55720 (0.0008) [2023-10-12 22:20:19,162][44959] Updated weights for policy 1, policy_version 55730 (0.0009) [2023-10-12 22:20:19,526][44959] Updated weights for policy 1, policy_version 55740 (0.0007) [2023-10-12 22:20:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113868800. Throughput: 0: 1642.4, 1: 1645.1. Samples: 28472122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:20:21,443][43579] Avg episode reward: [(0, '276.810'), (1, '274.360')] [2023-10-12 22:20:22,245][44958] Updated weights for policy 0, policy_version 55460 (0.0009) [2023-10-12 22:20:22,632][44958] Updated weights for policy 0, policy_version 55470 (0.0010) [2023-10-12 22:20:22,988][44958] Updated weights for policy 0, policy_version 55480 (0.0008) [2023-10-12 22:20:23,558][44959] Updated weights for policy 1, policy_version 55750 (0.0009) [2023-10-12 22:20:23,933][44959] Updated weights for policy 1, policy_version 55760 (0.0009) [2023-10-12 22:20:24,294][44959] Updated weights for policy 1, policy_version 55770 (0.0011) [2023-10-12 22:20:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113934336. Throughput: 0: 1634.6, 1: 1655.5. Samples: 28491732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:20:26,443][43579] Avg episode reward: [(0, '273.140'), (1, '273.650')] [2023-10-12 22:20:27,287][44958] Updated weights for policy 0, policy_version 55490 (0.0008) [2023-10-12 22:20:27,650][44958] Updated weights for policy 0, policy_version 55500 (0.0008) [2023-10-12 22:20:28,020][44958] Updated weights for policy 0, policy_version 55510 (0.0008) [2023-10-12 22:20:28,393][44958] Updated weights for policy 0, policy_version 55520 (0.0007) [2023-10-12 22:20:28,597][44959] Updated weights for policy 1, policy_version 55780 (0.0011) [2023-10-12 22:20:28,967][44959] Updated weights for policy 1, policy_version 55790 (0.0010) [2023-10-12 22:20:29,326][44959] Updated weights for policy 1, policy_version 55800 (0.0011) [2023-10-12 22:20:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 113999872. Throughput: 0: 1634.2, 1: 1647.6. Samples: 28511780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:20:31,444][43579] Avg episode reward: [(0, '276.960'), (1, '278.560')] [2023-10-12 22:20:32,389][44958] Updated weights for policy 0, policy_version 55530 (0.0007) [2023-10-12 22:20:32,758][44958] Updated weights for policy 0, policy_version 55540 (0.0008) [2023-10-12 22:20:33,122][44958] Updated weights for policy 0, policy_version 55550 (0.0008) [2023-10-12 22:20:33,372][44959] Updated weights for policy 1, policy_version 55810 (0.0009) [2023-10-12 22:20:33,745][44959] Updated weights for policy 1, policy_version 55820 (0.0009) [2023-10-12 22:20:34,106][44959] Updated weights for policy 1, policy_version 55830 (0.0010) [2023-10-12 22:20:34,475][44959] Updated weights for policy 1, policy_version 55840 (0.0011) [2023-10-12 22:20:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114065408. Throughput: 0: 1633.4, 1: 1648.2. Samples: 28521330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:20:36,443][43579] Avg episode reward: [(0, '277.050'), (1, '279.850')] [2023-10-12 22:20:37,463][44958] Updated weights for policy 0, policy_version 55560 (0.0008) [2023-10-12 22:20:37,830][44958] Updated weights for policy 0, policy_version 55570 (0.0007) [2023-10-12 22:20:38,206][44958] Updated weights for policy 0, policy_version 55580 (0.0009) [2023-10-12 22:20:38,533][44959] Updated weights for policy 1, policy_version 55850 (0.0008) [2023-10-12 22:20:38,902][44959] Updated weights for policy 1, policy_version 55860 (0.0008) [2023-10-12 22:20:39,271][44959] Updated weights for policy 1, policy_version 55870 (0.0011) [2023-10-12 22:20:41,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114130944. Throughput: 0: 1633.0, 1: 1656.2. Samples: 28541200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:20:41,443][43579] Avg episode reward: [(0, '276.360'), (1, '277.590')] [2023-10-12 22:20:42,218][44958] Updated weights for policy 0, policy_version 55590 (0.0010) [2023-10-12 22:20:42,597][44958] Updated weights for policy 0, policy_version 55600 (0.0010) [2023-10-12 22:20:42,957][44958] Updated weights for policy 0, policy_version 55610 (0.0008) [2023-10-12 22:20:43,470][44959] Updated weights for policy 1, policy_version 55880 (0.0008) [2023-10-12 22:20:43,830][44959] Updated weights for policy 1, policy_version 55890 (0.0007) [2023-10-12 22:20:44,203][44959] Updated weights for policy 1, policy_version 55900 (0.0007) [2023-10-12 22:20:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114196480. Throughput: 0: 1637.2, 1: 1649.7. Samples: 28561534. Policy #0 lag: (min: 4.0, avg: 9.7, max: 36.0) [2023-10-12 22:20:46,443][43579] Avg episode reward: [(0, '276.430'), (1, '276.950')] [2023-10-12 22:20:47,300][44958] Updated weights for policy 0, policy_version 55620 (0.0007) [2023-10-12 22:20:47,684][44958] Updated weights for policy 0, policy_version 55630 (0.0008) [2023-10-12 22:20:48,045][44958] Updated weights for policy 0, policy_version 55640 (0.0010) [2023-10-12 22:20:48,361][44959] Updated weights for policy 1, policy_version 55910 (0.0008) [2023-10-12 22:20:48,768][44959] Updated weights for policy 1, policy_version 55920 (0.0009) [2023-10-12 22:20:49,128][44959] Updated weights for policy 1, policy_version 55930 (0.0007) [2023-10-12 22:20:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114262016. Throughput: 0: 1632.7, 1: 1640.1. Samples: 28570416. Policy #0 lag: (min: 4.0, avg: 9.7, max: 36.0) [2023-10-12 22:20:51,443][43579] Avg episode reward: [(0, '279.710'), (1, '283.550')] [2023-10-12 22:20:52,262][44958] Updated weights for policy 0, policy_version 55650 (0.0009) [2023-10-12 22:20:52,636][44958] Updated weights for policy 0, policy_version 55660 (0.0007) [2023-10-12 22:20:53,009][44958] Updated weights for policy 0, policy_version 55670 (0.0008) [2023-10-12 22:20:53,317][44959] Updated weights for policy 1, policy_version 55940 (0.0008) [2023-10-12 22:20:53,380][44958] Updated weights for policy 0, policy_version 55680 (0.0010) [2023-10-12 22:20:53,685][44959] Updated weights for policy 1, policy_version 55950 (0.0009) [2023-10-12 22:20:54,048][44959] Updated weights for policy 1, policy_version 55960 (0.0009) [2023-10-12 22:20:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114327552. Throughput: 0: 1630.5, 1: 1649.6. Samples: 28590236. Policy #0 lag: (min: 4.0, avg: 9.7, max: 36.0) [2023-10-12 22:20:56,444][43579] Avg episode reward: [(0, '275.330'), (1, '282.710')] [2023-10-12 22:20:57,426][44958] Updated weights for policy 0, policy_version 55690 (0.0008) [2023-10-12 22:20:57,809][44958] Updated weights for policy 0, policy_version 55700 (0.0007) [2023-10-12 22:20:58,085][44959] Updated weights for policy 1, policy_version 55970 (0.0008) [2023-10-12 22:20:58,170][44958] Updated weights for policy 0, policy_version 55710 (0.0009) [2023-10-12 22:20:58,451][44959] Updated weights for policy 1, policy_version 55980 (0.0008) [2023-10-12 22:20:58,824][44959] Updated weights for policy 1, policy_version 55990 (0.0009) [2023-10-12 22:20:59,191][44959] Updated weights for policy 1, policy_version 56000 (0.0010) [2023-10-12 22:21:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114393088. Throughput: 0: 1634.9, 1: 1652.7. Samples: 28610536. Policy #0 lag: (min: 4.0, avg: 9.7, max: 36.0) [2023-10-12 22:21:01,443][43579] Avg episode reward: [(0, '275.820'), (1, '285.750')] [2023-10-12 22:21:02,346][44958] Updated weights for policy 0, policy_version 55720 (0.0009) [2023-10-12 22:21:02,726][44958] Updated weights for policy 0, policy_version 55730 (0.0008) [2023-10-12 22:21:03,101][44958] Updated weights for policy 0, policy_version 55740 (0.0007) [2023-10-12 22:21:03,430][44959] Updated weights for policy 1, policy_version 56010 (0.0008) [2023-10-12 22:21:03,797][44959] Updated weights for policy 1, policy_version 56020 (0.0009) [2023-10-12 22:21:04,176][44959] Updated weights for policy 1, policy_version 56030 (0.0007) [2023-10-12 22:21:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 114458624. Throughput: 0: 1637.6, 1: 1641.4. Samples: 28619678. Policy #0 lag: (min: 4.0, avg: 9.7, max: 36.0) [2023-10-12 22:21:06,444][43579] Avg episode reward: [(0, '276.760'), (1, '283.740')] [2023-10-12 22:21:07,380][44958] Updated weights for policy 0, policy_version 55750 (0.0008) [2023-10-12 22:21:07,750][44958] Updated weights for policy 0, policy_version 55760 (0.0007) [2023-10-12 22:21:08,115][44958] Updated weights for policy 0, policy_version 55770 (0.0007) [2023-10-12 22:21:08,460][44959] Updated weights for policy 1, policy_version 56040 (0.0009) [2023-10-12 22:21:08,832][44959] Updated weights for policy 1, policy_version 56050 (0.0009) [2023-10-12 22:21:09,198][44959] Updated weights for policy 1, policy_version 56060 (0.0009) [2023-10-12 22:21:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114524160. Throughput: 0: 1640.7, 1: 1646.7. Samples: 28639664. Policy #0 lag: (min: 4.0, avg: 9.7, max: 36.0) [2023-10-12 22:21:11,443][43579] Avg episode reward: [(0, '280.350'), (1, '283.440')] [2023-10-12 22:21:12,291][44958] Updated weights for policy 0, policy_version 55780 (0.0009) [2023-10-12 22:21:12,667][44958] Updated weights for policy 0, policy_version 55790 (0.0009) [2023-10-12 22:21:13,036][44958] Updated weights for policy 0, policy_version 55800 (0.0010) [2023-10-12 22:21:13,235][44959] Updated weights for policy 1, policy_version 56070 (0.0009) [2023-10-12 22:21:13,606][44959] Updated weights for policy 1, policy_version 56080 (0.0009) [2023-10-12 22:21:13,966][44959] Updated weights for policy 1, policy_version 56090 (0.0009) [2023-10-12 22:21:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 114589696. Throughput: 0: 1647.7, 1: 1652.9. Samples: 28660310. Policy #0 lag: (min: 4.0, avg: 9.7, max: 36.0) [2023-10-12 22:21:16,444][43579] Avg episode reward: [(0, '279.530'), (1, '284.330')] [2023-10-12 22:21:17,135][44958] Updated weights for policy 0, policy_version 55810 (0.0008) [2023-10-12 22:21:17,498][44958] Updated weights for policy 0, policy_version 55820 (0.0010) [2023-10-12 22:21:17,874][44958] Updated weights for policy 0, policy_version 55830 (0.0009) [2023-10-12 22:21:18,107][44959] Updated weights for policy 1, policy_version 56100 (0.0009) [2023-10-12 22:21:18,243][44958] Updated weights for policy 0, policy_version 55840 (0.0009) [2023-10-12 22:21:18,475][44959] Updated weights for policy 1, policy_version 56110 (0.0007) [2023-10-12 22:21:18,839][44959] Updated weights for policy 1, policy_version 56120 (0.0007) [2023-10-12 22:21:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114655232. Throughput: 0: 1643.9, 1: 1640.4. Samples: 28669128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:21:21,444][43579] Avg episode reward: [(0, '279.150'), (1, '282.130')] [2023-10-12 22:21:22,640][44958] Updated weights for policy 0, policy_version 55850 (0.0009) [2023-10-12 22:21:23,009][44958] Updated weights for policy 0, policy_version 55860 (0.0008) [2023-10-12 22:21:23,048][44959] Updated weights for policy 1, policy_version 56130 (0.0008) [2023-10-12 22:21:23,387][44958] Updated weights for policy 0, policy_version 55870 (0.0007) [2023-10-12 22:21:23,409][44959] Updated weights for policy 1, policy_version 56140 (0.0008) [2023-10-12 22:21:23,784][44959] Updated weights for policy 1, policy_version 56150 (0.0008) [2023-10-12 22:21:24,149][44959] Updated weights for policy 1, policy_version 56160 (0.0007) [2023-10-12 22:21:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114720768. Throughput: 0: 1634.8, 1: 1651.6. Samples: 28689090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:21:26,443][43579] Avg episode reward: [(0, '283.440'), (1, '278.710')] [2023-10-12 22:21:27,678][44958] Updated weights for policy 0, policy_version 55880 (0.0008) [2023-10-12 22:21:28,032][44959] Updated weights for policy 1, policy_version 56170 (0.0008) [2023-10-12 22:21:28,049][44958] Updated weights for policy 0, policy_version 55890 (0.0007) [2023-10-12 22:21:28,398][44959] Updated weights for policy 1, policy_version 56180 (0.0008) [2023-10-12 22:21:28,422][44958] Updated weights for policy 0, policy_version 55900 (0.0009) [2023-10-12 22:21:28,763][44959] Updated weights for policy 1, policy_version 56190 (0.0009) [2023-10-12 22:21:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 114786304. Throughput: 0: 1630.8, 1: 1654.9. Samples: 28709390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:21:31,443][43579] Avg episode reward: [(0, '279.650'), (1, '274.570')] [2023-10-12 22:21:32,644][44958] Updated weights for policy 0, policy_version 55910 (0.0008) [2023-10-12 22:21:32,928][44959] Updated weights for policy 1, policy_version 56200 (0.0008) [2023-10-12 22:21:33,016][44958] Updated weights for policy 0, policy_version 55920 (0.0007) [2023-10-12 22:21:33,293][44959] Updated weights for policy 1, policy_version 56210 (0.0009) [2023-10-12 22:21:33,387][44958] Updated weights for policy 0, policy_version 55930 (0.0008) [2023-10-12 22:21:33,660][44959] Updated weights for policy 1, policy_version 56220 (0.0010) [2023-10-12 22:21:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114851840. Throughput: 0: 1630.6, 1: 1650.3. Samples: 28718056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:21:36,444][43579] Avg episode reward: [(0, '278.910'), (1, '274.520')] [2023-10-12 22:21:37,522][44958] Updated weights for policy 0, policy_version 55940 (0.0007) [2023-10-12 22:21:37,896][44958] Updated weights for policy 0, policy_version 55950 (0.0008) [2023-10-12 22:21:38,082][44959] Updated weights for policy 1, policy_version 56230 (0.0008) [2023-10-12 22:21:38,266][44958] Updated weights for policy 0, policy_version 55960 (0.0008) [2023-10-12 22:21:38,483][44959] Updated weights for policy 1, policy_version 56240 (0.0007) [2023-10-12 22:21:38,850][44959] Updated weights for policy 1, policy_version 56250 (0.0010) [2023-10-12 22:21:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 114917376. Throughput: 0: 1635.3, 1: 1655.5. Samples: 28738320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:21:41,443][43579] Avg episode reward: [(0, '278.580'), (1, '277.200')] [2023-10-12 22:21:42,619][44958] Updated weights for policy 0, policy_version 55970 (0.0009) [2023-10-12 22:21:43,015][44958] Updated weights for policy 0, policy_version 55980 (0.0008) [2023-10-12 22:21:43,079][44959] Updated weights for policy 1, policy_version 56260 (0.0008) [2023-10-12 22:21:43,381][44958] Updated weights for policy 0, policy_version 55990 (0.0010) [2023-10-12 22:21:43,439][44959] Updated weights for policy 1, policy_version 56270 (0.0007) [2023-10-12 22:21:43,755][44958] Updated weights for policy 0, policy_version 56000 (0.0010) [2023-10-12 22:21:43,814][44959] Updated weights for policy 1, policy_version 56280 (0.0008) [2023-10-12 22:21:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 114982912. Throughput: 0: 1630.4, 1: 1657.5. Samples: 28758494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:21:46,444][43579] Avg episode reward: [(0, '281.740'), (1, '278.030')] [2023-10-12 22:21:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000056000_57344000.pth... [2023-10-12 22:21:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000056288_57638912.pth... [2023-10-12 22:21:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000054464_55771136.pth [2023-10-12 22:21:46,498][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000054752_56066048.pth [2023-10-12 22:21:47,737][44958] Updated weights for policy 0, policy_version 56010 (0.0009) [2023-10-12 22:21:47,871][44959] Updated weights for policy 1, policy_version 56290 (0.0008) [2023-10-12 22:21:48,106][44958] Updated weights for policy 0, policy_version 56020 (0.0008) [2023-10-12 22:21:48,238][44959] Updated weights for policy 1, policy_version 56300 (0.0008) [2023-10-12 22:21:48,468][44958] Updated weights for policy 0, policy_version 56030 (0.0010) [2023-10-12 22:21:48,608][44959] Updated weights for policy 1, policy_version 56310 (0.0010) [2023-10-12 22:21:48,980][44959] Updated weights for policy 1, policy_version 56320 (0.0010) [2023-10-12 22:21:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115048448. Throughput: 0: 1630.7, 1: 1653.6. Samples: 28767472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:21:51,443][43579] Avg episode reward: [(0, '279.300'), (1, '278.150')] [2023-10-12 22:21:52,839][44958] Updated weights for policy 0, policy_version 56040 (0.0007) [2023-10-12 22:21:53,087][44959] Updated weights for policy 1, policy_version 56330 (0.0009) [2023-10-12 22:21:53,212][44958] Updated weights for policy 0, policy_version 56050 (0.0007) [2023-10-12 22:21:53,443][44959] Updated weights for policy 1, policy_version 56340 (0.0007) [2023-10-12 22:21:53,574][44958] Updated weights for policy 0, policy_version 56060 (0.0007) [2023-10-12 22:21:53,811][44959] Updated weights for policy 1, policy_version 56350 (0.0009) [2023-10-12 22:21:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115113984. Throughput: 0: 1630.0, 1: 1656.4. Samples: 28787556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:21:56,444][43579] Avg episode reward: [(0, '276.320'), (1, '280.130')] [2023-10-12 22:21:57,641][44958] Updated weights for policy 0, policy_version 56070 (0.0008) [2023-10-12 22:21:58,011][44958] Updated weights for policy 0, policy_version 56080 (0.0007) [2023-10-12 22:21:58,016][44959] Updated weights for policy 1, policy_version 56360 (0.0009) [2023-10-12 22:21:58,371][44958] Updated weights for policy 0, policy_version 56090 (0.0008) [2023-10-12 22:21:58,380][44959] Updated weights for policy 1, policy_version 56370 (0.0008) [2023-10-12 22:21:58,751][44959] Updated weights for policy 1, policy_version 56380 (0.0009) [2023-10-12 22:22:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115179520. Throughput: 0: 1624.0, 1: 1655.9. Samples: 28807906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:01,444][43579] Avg episode reward: [(0, '280.590'), (1, '288.630')] [2023-10-12 22:22:02,756][44958] Updated weights for policy 0, policy_version 56100 (0.0008) [2023-10-12 22:22:02,901][44959] Updated weights for policy 1, policy_version 56390 (0.0010) [2023-10-12 22:22:03,124][44958] Updated weights for policy 0, policy_version 56110 (0.0008) [2023-10-12 22:22:03,269][44959] Updated weights for policy 1, policy_version 56400 (0.0008) [2023-10-12 22:22:03,507][44958] Updated weights for policy 0, policy_version 56120 (0.0009) [2023-10-12 22:22:03,633][44959] Updated weights for policy 1, policy_version 56410 (0.0007) [2023-10-12 22:22:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115245056. Throughput: 0: 1626.8, 1: 1650.7. Samples: 28816614. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:06,444][43579] Avg episode reward: [(0, '278.820'), (1, '291.330')] [2023-10-12 22:22:07,564][44958] Updated weights for policy 0, policy_version 56130 (0.0009) [2023-10-12 22:22:07,816][44959] Updated weights for policy 1, policy_version 56420 (0.0007) [2023-10-12 22:22:07,933][44958] Updated weights for policy 0, policy_version 56140 (0.0009) [2023-10-12 22:22:08,187][44959] Updated weights for policy 1, policy_version 56430 (0.0009) [2023-10-12 22:22:08,309][44958] Updated weights for policy 0, policy_version 56150 (0.0009) [2023-10-12 22:22:08,553][44959] Updated weights for policy 1, policy_version 56440 (0.0007) [2023-10-12 22:22:08,674][44958] Updated weights for policy 0, policy_version 56160 (0.0007) [2023-10-12 22:22:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115310592. Throughput: 0: 1636.2, 1: 1653.3. Samples: 28837118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:11,444][43579] Avg episode reward: [(0, '279.670'), (1, '289.990')] [2023-10-12 22:22:12,731][44959] Updated weights for policy 1, policy_version 56450 (0.0009) [2023-10-12 22:22:12,782][44958] Updated weights for policy 0, policy_version 56170 (0.0008) [2023-10-12 22:22:13,094][44959] Updated weights for policy 1, policy_version 56460 (0.0009) [2023-10-12 22:22:13,156][44958] Updated weights for policy 0, policy_version 56180 (0.0010) [2023-10-12 22:22:13,464][44959] Updated weights for policy 1, policy_version 56470 (0.0008) [2023-10-12 22:22:13,519][44958] Updated weights for policy 0, policy_version 56190 (0.0008) [2023-10-12 22:22:13,830][44959] Updated weights for policy 1, policy_version 56480 (0.0010) [2023-10-12 22:22:16,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115376128. Throughput: 0: 1636.0, 1: 1655.0. Samples: 28857484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:16,443][43579] Avg episode reward: [(0, '274.800'), (1, '288.300')] [2023-10-12 22:22:17,741][44958] Updated weights for policy 0, policy_version 56200 (0.0007) [2023-10-12 22:22:17,768][44959] Updated weights for policy 1, policy_version 56490 (0.0007) [2023-10-12 22:22:18,114][44958] Updated weights for policy 0, policy_version 56210 (0.0008) [2023-10-12 22:22:18,128][44959] Updated weights for policy 1, policy_version 56500 (0.0008) [2023-10-12 22:22:18,487][44958] Updated weights for policy 0, policy_version 56220 (0.0008) [2023-10-12 22:22:18,499][44959] Updated weights for policy 1, policy_version 56510 (0.0009) [2023-10-12 22:22:21,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115441664. Throughput: 0: 1639.4, 1: 1654.8. Samples: 28866292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:21,444][43579] Avg episode reward: [(0, '273.930'), (1, '289.570')] [2023-10-12 22:22:22,573][44958] Updated weights for policy 0, policy_version 56230 (0.0008) [2023-10-12 22:22:22,637][44959] Updated weights for policy 1, policy_version 56520 (0.0008) [2023-10-12 22:22:22,947][44958] Updated weights for policy 0, policy_version 56240 (0.0008) [2023-10-12 22:22:23,003][44959] Updated weights for policy 1, policy_version 56530 (0.0007) [2023-10-12 22:22:23,312][44958] Updated weights for policy 0, policy_version 56250 (0.0009) [2023-10-12 22:22:23,373][44959] Updated weights for policy 1, policy_version 56540 (0.0007) [2023-10-12 22:22:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115507200. Throughput: 0: 1636.3, 1: 1657.1. Samples: 28886524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:26,443][43579] Avg episode reward: [(0, '276.510'), (1, '289.210')] [2023-10-12 22:22:27,650][44958] Updated weights for policy 0, policy_version 56260 (0.0009) [2023-10-12 22:22:27,799][44959] Updated weights for policy 1, policy_version 56550 (0.0009) [2023-10-12 22:22:28,023][44958] Updated weights for policy 0, policy_version 56270 (0.0008) [2023-10-12 22:22:28,199][44959] Updated weights for policy 1, policy_version 56560 (0.0009) [2023-10-12 22:22:28,392][44958] Updated weights for policy 0, policy_version 56280 (0.0008) [2023-10-12 22:22:28,559][44959] Updated weights for policy 1, policy_version 56570 (0.0008) [2023-10-12 22:22:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 115572736. Throughput: 0: 1637.0, 1: 1647.6. Samples: 28906300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:31,444][43579] Avg episode reward: [(0, '277.570'), (1, '284.220')] [2023-10-12 22:22:32,635][44958] Updated weights for policy 0, policy_version 56290 (0.0008) [2023-10-12 22:22:32,833][44959] Updated weights for policy 1, policy_version 56580 (0.0008) [2023-10-12 22:22:33,034][44958] Updated weights for policy 0, policy_version 56300 (0.0008) [2023-10-12 22:22:33,194][44959] Updated weights for policy 1, policy_version 56590 (0.0009) [2023-10-12 22:22:33,408][44958] Updated weights for policy 0, policy_version 56310 (0.0008) [2023-10-12 22:22:33,569][44959] Updated weights for policy 1, policy_version 56600 (0.0010) [2023-10-12 22:22:33,770][44958] Updated weights for policy 0, policy_version 56320 (0.0009) [2023-10-12 22:22:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115638272. Throughput: 0: 1634.3, 1: 1645.3. Samples: 28915054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:36,444][43579] Avg episode reward: [(0, '276.080'), (1, '282.340')] [2023-10-12 22:22:37,794][44958] Updated weights for policy 0, policy_version 56330 (0.0009) [2023-10-12 22:22:37,865][44959] Updated weights for policy 1, policy_version 56610 (0.0010) [2023-10-12 22:22:38,160][44958] Updated weights for policy 0, policy_version 56340 (0.0009) [2023-10-12 22:22:38,240][44959] Updated weights for policy 1, policy_version 56620 (0.0010) [2023-10-12 22:22:38,528][44958] Updated weights for policy 0, policy_version 56350 (0.0009) [2023-10-12 22:22:38,601][44959] Updated weights for policy 1, policy_version 56630 (0.0008) [2023-10-12 22:22:38,966][44959] Updated weights for policy 1, policy_version 56640 (0.0010) [2023-10-12 22:22:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115703808. Throughput: 0: 1637.0, 1: 1645.2. Samples: 28935252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:41,444][43579] Avg episode reward: [(0, '279.070'), (1, '280.860')] [2023-10-12 22:22:42,840][44958] Updated weights for policy 0, policy_version 56360 (0.0010) [2023-10-12 22:22:43,166][44959] Updated weights for policy 1, policy_version 56650 (0.0008) [2023-10-12 22:22:43,212][44958] Updated weights for policy 0, policy_version 56370 (0.0008) [2023-10-12 22:22:43,542][44959] Updated weights for policy 1, policy_version 56660 (0.0008) [2023-10-12 22:22:43,588][44958] Updated weights for policy 0, policy_version 56380 (0.0009) [2023-10-12 22:22:43,910][44959] Updated weights for policy 1, policy_version 56670 (0.0008) [2023-10-12 22:22:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115769344. Throughput: 0: 1635.7, 1: 1637.9. Samples: 28955218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:46,444][43579] Avg episode reward: [(0, '285.850'), (1, '280.590')] [2023-10-12 22:22:47,697][44958] Updated weights for policy 0, policy_version 56390 (0.0008) [2023-10-12 22:22:48,051][44959] Updated weights for policy 1, policy_version 56680 (0.0007) [2023-10-12 22:22:48,065][44958] Updated weights for policy 0, policy_version 56400 (0.0008) [2023-10-12 22:22:48,418][44959] Updated weights for policy 1, policy_version 56690 (0.0009) [2023-10-12 22:22:48,439][44958] Updated weights for policy 0, policy_version 56410 (0.0009) [2023-10-12 22:22:48,778][44959] Updated weights for policy 1, policy_version 56700 (0.0008) [2023-10-12 22:22:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115834880. Throughput: 0: 1636.1, 1: 1640.9. Samples: 28964078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:51,443][43579] Avg episode reward: [(0, '284.790'), (1, '281.400')] [2023-10-12 22:22:52,683][44958] Updated weights for policy 0, policy_version 56420 (0.0007) [2023-10-12 22:22:52,960][44959] Updated weights for policy 1, policy_version 56710 (0.0008) [2023-10-12 22:22:53,065][44958] Updated weights for policy 0, policy_version 56430 (0.0007) [2023-10-12 22:22:53,336][44959] Updated weights for policy 1, policy_version 56720 (0.0009) [2023-10-12 22:22:53,430][44958] Updated weights for policy 0, policy_version 56440 (0.0007) [2023-10-12 22:22:53,702][44959] Updated weights for policy 1, policy_version 56730 (0.0009) [2023-10-12 22:22:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 115900416. Throughput: 0: 1636.5, 1: 1636.4. Samples: 28984396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:22:56,443][43579] Avg episode reward: [(0, '286.360'), (1, '281.620')] [2023-10-12 22:22:57,494][44958] Updated weights for policy 0, policy_version 56450 (0.0008) [2023-10-12 22:22:57,712][44959] Updated weights for policy 1, policy_version 56740 (0.0010) [2023-10-12 22:22:57,865][44958] Updated weights for policy 0, policy_version 56460 (0.0008) [2023-10-12 22:22:58,082][44959] Updated weights for policy 1, policy_version 56750 (0.0007) [2023-10-12 22:22:58,235][44958] Updated weights for policy 0, policy_version 56470 (0.0007) [2023-10-12 22:22:58,444][44959] Updated weights for policy 1, policy_version 56760 (0.0008) [2023-10-12 22:22:58,606][44958] Updated weights for policy 0, policy_version 56480 (0.0008) [2023-10-12 22:23:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 115965952. Throughput: 0: 1638.0, 1: 1633.6. Samples: 29004708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:23:01,443][43579] Avg episode reward: [(0, '284.120'), (1, '285.670')] [2023-10-12 22:23:02,558][44959] Updated weights for policy 1, policy_version 56770 (0.0009) [2023-10-12 22:23:02,749][44958] Updated weights for policy 0, policy_version 56490 (0.0008) [2023-10-12 22:23:02,928][44959] Updated weights for policy 1, policy_version 56780 (0.0009) [2023-10-12 22:23:03,117][44958] Updated weights for policy 0, policy_version 56500 (0.0007) [2023-10-12 22:23:03,290][44959] Updated weights for policy 1, policy_version 56790 (0.0008) [2023-10-12 22:23:03,489][44958] Updated weights for policy 0, policy_version 56510 (0.0008) [2023-10-12 22:23:03,656][44959] Updated weights for policy 1, policy_version 56800 (0.0010) [2023-10-12 22:23:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116031488. Throughput: 0: 1640.2, 1: 1634.4. Samples: 29013646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:23:06,443][43579] Avg episode reward: [(0, '282.370'), (1, '281.040')] [2023-10-12 22:23:07,709][44958] Updated weights for policy 0, policy_version 56520 (0.0007) [2023-10-12 22:23:07,937][44959] Updated weights for policy 1, policy_version 56810 (0.0007) [2023-10-12 22:23:08,084][44958] Updated weights for policy 0, policy_version 56530 (0.0008) [2023-10-12 22:23:08,300][44959] Updated weights for policy 1, policy_version 56820 (0.0009) [2023-10-12 22:23:08,454][44958] Updated weights for policy 0, policy_version 56540 (0.0007) [2023-10-12 22:23:08,660][44959] Updated weights for policy 1, policy_version 56830 (0.0009) [2023-10-12 22:23:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116097024. Throughput: 0: 1640.0, 1: 1628.0. Samples: 29033588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:23:11,444][43579] Avg episode reward: [(0, '274.650'), (1, '281.590')] [2023-10-12 22:23:12,797][44958] Updated weights for policy 0, policy_version 56550 (0.0009) [2023-10-12 22:23:13,163][44959] Updated weights for policy 1, policy_version 56840 (0.0008) [2023-10-12 22:23:13,171][44958] Updated weights for policy 0, policy_version 56560 (0.0009) [2023-10-12 22:23:13,534][44959] Updated weights for policy 1, policy_version 56850 (0.0007) [2023-10-12 22:23:13,541][44958] Updated weights for policy 0, policy_version 56570 (0.0008) [2023-10-12 22:23:13,898][44959] Updated weights for policy 1, policy_version 56860 (0.0007) [2023-10-12 22:23:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116162560. Throughput: 0: 1641.7, 1: 1637.6. Samples: 29053866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:23:16,443][43579] Avg episode reward: [(0, '276.850'), (1, '279.050')] [2023-10-12 22:23:17,530][44958] Updated weights for policy 0, policy_version 56580 (0.0008) [2023-10-12 22:23:17,908][44958] Updated weights for policy 0, policy_version 56590 (0.0008) [2023-10-12 22:23:17,915][44959] Updated weights for policy 1, policy_version 56870 (0.0009) [2023-10-12 22:23:18,276][44958] Updated weights for policy 0, policy_version 56600 (0.0009) [2023-10-12 22:23:18,283][44959] Updated weights for policy 1, policy_version 56880 (0.0008) [2023-10-12 22:23:18,662][44959] Updated weights for policy 1, policy_version 56890 (0.0009) [2023-10-12 22:23:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116228096. Throughput: 0: 1643.4, 1: 1637.4. Samples: 29062690. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 22:23:21,443][43579] Avg episode reward: [(0, '277.120'), (1, '277.450')] [2023-10-12 22:23:22,582][44958] Updated weights for policy 0, policy_version 56610 (0.0008) [2023-10-12 22:23:22,686][44959] Updated weights for policy 1, policy_version 56900 (0.0010) [2023-10-12 22:23:22,948][44958] Updated weights for policy 0, policy_version 56620 (0.0007) [2023-10-12 22:23:23,051][44959] Updated weights for policy 1, policy_version 56910 (0.0007) [2023-10-12 22:23:23,325][44958] Updated weights for policy 0, policy_version 56630 (0.0007) [2023-10-12 22:23:23,413][44959] Updated weights for policy 1, policy_version 56920 (0.0007) [2023-10-12 22:23:23,696][44958] Updated weights for policy 0, policy_version 56640 (0.0008) [2023-10-12 22:23:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116293632. Throughput: 0: 1638.2, 1: 1641.3. Samples: 29082828. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 22:23:26,443][43579] Avg episode reward: [(0, '275.090'), (1, '273.000')] [2023-10-12 22:23:27,499][44959] Updated weights for policy 1, policy_version 56930 (0.0009) [2023-10-12 22:23:27,867][44959] Updated weights for policy 1, policy_version 56940 (0.0008) [2023-10-12 22:23:27,922][44958] Updated weights for policy 0, policy_version 56650 (0.0008) [2023-10-12 22:23:28,245][44959] Updated weights for policy 1, policy_version 56950 (0.0010) [2023-10-12 22:23:28,294][44958] Updated weights for policy 0, policy_version 56660 (0.0008) [2023-10-12 22:23:28,615][44959] Updated weights for policy 1, policy_version 56960 (0.0010) [2023-10-12 22:23:28,667][44958] Updated weights for policy 0, policy_version 56670 (0.0009) [2023-10-12 22:23:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116359168. Throughput: 0: 1644.1, 1: 1644.1. Samples: 29103190. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 22:23:31,443][43579] Avg episode reward: [(0, '276.220'), (1, '272.660')] [2023-10-12 22:23:32,763][44959] Updated weights for policy 1, policy_version 56970 (0.0007) [2023-10-12 22:23:32,808][44958] Updated weights for policy 0, policy_version 56680 (0.0007) [2023-10-12 22:23:33,130][44959] Updated weights for policy 1, policy_version 56980 (0.0007) [2023-10-12 22:23:33,175][44958] Updated weights for policy 0, policy_version 56690 (0.0007) [2023-10-12 22:23:33,497][44959] Updated weights for policy 1, policy_version 56990 (0.0008) [2023-10-12 22:23:33,544][44958] Updated weights for policy 0, policy_version 56700 (0.0008) [2023-10-12 22:23:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116424704. Throughput: 0: 1644.2, 1: 1643.2. Samples: 29112014. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 22:23:36,443][43579] Avg episode reward: [(0, '278.680'), (1, '280.200')] [2023-10-12 22:23:37,638][44959] Updated weights for policy 1, policy_version 57000 (0.0007) [2023-10-12 22:23:37,752][44958] Updated weights for policy 0, policy_version 56710 (0.0008) [2023-10-12 22:23:38,007][44959] Updated weights for policy 1, policy_version 57010 (0.0007) [2023-10-12 22:23:38,130][44958] Updated weights for policy 0, policy_version 56720 (0.0010) [2023-10-12 22:23:38,373][44959] Updated weights for policy 1, policy_version 57020 (0.0010) [2023-10-12 22:23:38,501][44958] Updated weights for policy 0, policy_version 56730 (0.0008) [2023-10-12 22:23:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116490240. Throughput: 0: 1641.1, 1: 1645.7. Samples: 29132302. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 22:23:41,443][43579] Avg episode reward: [(0, '280.790'), (1, '278.400')] [2023-10-12 22:23:42,419][44959] Updated weights for policy 1, policy_version 57030 (0.0007) [2023-10-12 22:23:42,443][44958] Updated weights for policy 0, policy_version 56740 (0.0008) [2023-10-12 22:23:42,779][44959] Updated weights for policy 1, policy_version 57040 (0.0007) [2023-10-12 22:23:42,813][44958] Updated weights for policy 0, policy_version 56750 (0.0007) [2023-10-12 22:23:43,150][44959] Updated weights for policy 1, policy_version 57050 (0.0009) [2023-10-12 22:23:43,181][44958] Updated weights for policy 0, policy_version 56760 (0.0007) [2023-10-12 22:23:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116555776. Throughput: 0: 1650.2, 1: 1647.9. Samples: 29153122. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 22:23:46,443][43579] Avg episode reward: [(0, '282.340'), (1, '275.240')] [2023-10-12 22:23:46,453][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000056768_58130432.pth... [2023-10-12 22:23:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000057056_58425344.pth... [2023-10-12 22:23:46,483][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000055232_56557568.pth [2023-10-12 22:23:46,499][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000055520_56852480.pth [2023-10-12 22:23:47,244][44958] Updated weights for policy 0, policy_version 56770 (0.0009) [2023-10-12 22:23:47,415][44959] Updated weights for policy 1, policy_version 57060 (0.0008) [2023-10-12 22:23:47,618][44958] Updated weights for policy 0, policy_version 56780 (0.0008) [2023-10-12 22:23:47,779][44959] Updated weights for policy 1, policy_version 57070 (0.0009) [2023-10-12 22:23:47,984][44958] Updated weights for policy 0, policy_version 56790 (0.0007) [2023-10-12 22:23:48,154][44959] Updated weights for policy 1, policy_version 57080 (0.0008) [2023-10-12 22:23:48,361][44958] Updated weights for policy 0, policy_version 56800 (0.0008) [2023-10-12 22:23:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116621312. Throughput: 0: 1644.1, 1: 1642.4. Samples: 29161536. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 22:23:51,443][43579] Avg episode reward: [(0, '279.630'), (1, '279.010')] [2023-10-12 22:23:52,347][44959] Updated weights for policy 1, policy_version 57090 (0.0007) [2023-10-12 22:23:52,700][44958] Updated weights for policy 0, policy_version 56810 (0.0008) [2023-10-12 22:23:52,706][44959] Updated weights for policy 1, policy_version 57100 (0.0007) [2023-10-12 22:23:53,075][44958] Updated weights for policy 0, policy_version 56820 (0.0008) [2023-10-12 22:23:53,085][44959] Updated weights for policy 1, policy_version 57110 (0.0007) [2023-10-12 22:23:53,438][44958] Updated weights for policy 0, policy_version 56830 (0.0008) [2023-10-12 22:23:53,452][44959] Updated weights for policy 1, policy_version 57120 (0.0008) [2023-10-12 22:23:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116686848. Throughput: 0: 1644.8, 1: 1649.3. Samples: 29181822. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-12 22:23:56,443][43579] Avg episode reward: [(0, '281.680'), (1, '280.340')] [2023-10-12 22:23:57,649][44958] Updated weights for policy 0, policy_version 56840 (0.0008) [2023-10-12 22:23:57,718][44959] Updated weights for policy 1, policy_version 57130 (0.0010) [2023-10-12 22:23:58,014][44958] Updated weights for policy 0, policy_version 56850 (0.0007) [2023-10-12 22:23:58,092][44959] Updated weights for policy 1, policy_version 57140 (0.0008) [2023-10-12 22:23:58,393][44958] Updated weights for policy 0, policy_version 56860 (0.0009) [2023-10-12 22:23:58,467][44959] Updated weights for policy 1, policy_version 57150 (0.0010) [2023-10-12 22:24:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116752384. Throughput: 0: 1645.8, 1: 1647.5. Samples: 29202064. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) [2023-10-12 22:24:01,444][43579] Avg episode reward: [(0, '282.110'), (1, '274.790')] [2023-10-12 22:24:02,651][44958] Updated weights for policy 0, policy_version 56870 (0.0009) [2023-10-12 22:24:02,668][44959] Updated weights for policy 1, policy_version 57160 (0.0007) [2023-10-12 22:24:03,023][44958] Updated weights for policy 0, policy_version 56880 (0.0007) [2023-10-12 22:24:03,044][44959] Updated weights for policy 1, policy_version 57170 (0.0008) [2023-10-12 22:24:03,387][44958] Updated weights for policy 0, policy_version 56890 (0.0008) [2023-10-12 22:24:03,413][44959] Updated weights for policy 1, policy_version 57180 (0.0008) [2023-10-12 22:24:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116817920. Throughput: 0: 1641.1, 1: 1646.6. Samples: 29210636. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) [2023-10-12 22:24:06,444][43579] Avg episode reward: [(0, '277.480'), (1, '268.040')] [2023-10-12 22:24:07,639][44959] Updated weights for policy 1, policy_version 57190 (0.0008) [2023-10-12 22:24:07,699][44958] Updated weights for policy 0, policy_version 56900 (0.0008) [2023-10-12 22:24:08,010][44959] Updated weights for policy 1, policy_version 57200 (0.0010) [2023-10-12 22:24:08,081][44958] Updated weights for policy 0, policy_version 56910 (0.0008) [2023-10-12 22:24:08,378][44959] Updated weights for policy 1, policy_version 57210 (0.0007) [2023-10-12 22:24:08,461][44958] Updated weights for policy 0, policy_version 56920 (0.0009) [2023-10-12 22:24:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116883456. Throughput: 0: 1636.8, 1: 1653.6. Samples: 29230898. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) [2023-10-12 22:24:11,444][43579] Avg episode reward: [(0, '280.760'), (1, '273.690')] [2023-10-12 22:24:12,521][44959] Updated weights for policy 1, policy_version 57220 (0.0007) [2023-10-12 22:24:12,561][44958] Updated weights for policy 0, policy_version 56930 (0.0008) [2023-10-12 22:24:12,890][44959] Updated weights for policy 1, policy_version 57230 (0.0008) [2023-10-12 22:24:12,945][44958] Updated weights for policy 0, policy_version 56940 (0.0009) [2023-10-12 22:24:13,255][44959] Updated weights for policy 1, policy_version 57240 (0.0009) [2023-10-12 22:24:13,309][44958] Updated weights for policy 0, policy_version 56950 (0.0007) [2023-10-12 22:24:13,675][44958] Updated weights for policy 0, policy_version 56960 (0.0009) [2023-10-12 22:24:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 116948992. Throughput: 0: 1638.3, 1: 1650.5. Samples: 29251186. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) [2023-10-12 22:24:16,443][43579] Avg episode reward: [(0, '280.550'), (1, '272.070')] [2023-10-12 22:24:17,621][44959] Updated weights for policy 1, policy_version 57250 (0.0010) [2023-10-12 22:24:17,865][44958] Updated weights for policy 0, policy_version 56970 (0.0008) [2023-10-12 22:24:17,977][44959] Updated weights for policy 1, policy_version 57260 (0.0007) [2023-10-12 22:24:18,236][44958] Updated weights for policy 0, policy_version 56980 (0.0008) [2023-10-12 22:24:18,339][44959] Updated weights for policy 1, policy_version 57270 (0.0008) [2023-10-12 22:24:18,618][44958] Updated weights for policy 0, policy_version 56990 (0.0009) [2023-10-12 22:24:18,713][44959] Updated weights for policy 1, policy_version 57280 (0.0007) [2023-10-12 22:24:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117014528. Throughput: 0: 1637.2, 1: 1647.6. Samples: 29259834. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) [2023-10-12 22:24:21,443][43579] Avg episode reward: [(0, '285.880'), (1, '274.490')] [2023-10-12 22:24:22,787][44959] Updated weights for policy 1, policy_version 57290 (0.0009) [2023-10-12 22:24:22,893][44958] Updated weights for policy 0, policy_version 57000 (0.0009) [2023-10-12 22:24:23,158][44959] Updated weights for policy 1, policy_version 57300 (0.0008) [2023-10-12 22:24:23,262][44958] Updated weights for policy 0, policy_version 57010 (0.0008) [2023-10-12 22:24:23,517][44959] Updated weights for policy 1, policy_version 57310 (0.0008) [2023-10-12 22:24:23,631][44958] Updated weights for policy 0, policy_version 57020 (0.0009) [2023-10-12 22:24:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117080064. Throughput: 0: 1633.4, 1: 1648.5. Samples: 29279986. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) [2023-10-12 22:24:26,444][43579] Avg episode reward: [(0, '285.790'), (1, '272.970')] [2023-10-12 22:24:27,552][44959] Updated weights for policy 1, policy_version 57320 (0.0008) [2023-10-12 22:24:27,759][44958] Updated weights for policy 0, policy_version 57030 (0.0007) [2023-10-12 22:24:27,926][44959] Updated weights for policy 1, policy_version 57330 (0.0009) [2023-10-12 22:24:28,132][44958] Updated weights for policy 0, policy_version 57040 (0.0007) [2023-10-12 22:24:28,287][44959] Updated weights for policy 1, policy_version 57340 (0.0009) [2023-10-12 22:24:28,513][44958] Updated weights for policy 0, policy_version 57050 (0.0009) [2023-10-12 22:24:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117145600. Throughput: 0: 1627.3, 1: 1645.1. Samples: 29300378. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) [2023-10-12 22:24:31,443][43579] Avg episode reward: [(0, '283.620'), (1, '280.310')] [2023-10-12 22:24:32,225][44959] Updated weights for policy 1, policy_version 57350 (0.0007) [2023-10-12 22:24:32,596][44959] Updated weights for policy 1, policy_version 57360 (0.0007) [2023-10-12 22:24:32,627][44958] Updated weights for policy 0, policy_version 57060 (0.0008) [2023-10-12 22:24:32,970][44959] Updated weights for policy 1, policy_version 57370 (0.0008) [2023-10-12 22:24:33,001][44958] Updated weights for policy 0, policy_version 57070 (0.0009) [2023-10-12 22:24:33,375][44958] Updated weights for policy 0, policy_version 57080 (0.0008) [2023-10-12 22:24:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 117211136. Throughput: 0: 1631.3, 1: 1650.5. Samples: 29309218. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) [2023-10-12 22:24:36,444][43579] Avg episode reward: [(0, '284.790'), (1, '283.470')] [2023-10-12 22:24:37,205][44959] Updated weights for policy 1, policy_version 57380 (0.0008) [2023-10-12 22:24:37,570][44959] Updated weights for policy 1, policy_version 57390 (0.0010) [2023-10-12 22:24:37,688][44958] Updated weights for policy 0, policy_version 57090 (0.0007) [2023-10-12 22:24:37,934][44959] Updated weights for policy 1, policy_version 57400 (0.0008) [2023-10-12 22:24:38,069][44958] Updated weights for policy 0, policy_version 57100 (0.0008) [2023-10-12 22:24:38,442][44958] Updated weights for policy 0, policy_version 57110 (0.0009) [2023-10-12 22:24:38,811][44958] Updated weights for policy 0, policy_version 57120 (0.0009) [2023-10-12 22:24:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117276672. Throughput: 0: 1627.3, 1: 1653.1. Samples: 29329438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:24:41,443][43579] Avg episode reward: [(0, '282.630'), (1, '285.400')] [2023-10-12 22:24:42,122][44959] Updated weights for policy 1, policy_version 57410 (0.0008) [2023-10-12 22:24:42,529][44959] Updated weights for policy 1, policy_version 57420 (0.0008) [2023-10-12 22:24:42,897][44959] Updated weights for policy 1, policy_version 57430 (0.0008) [2023-10-12 22:24:43,063][44958] Updated weights for policy 0, policy_version 57130 (0.0007) [2023-10-12 22:24:43,259][44959] Updated weights for policy 1, policy_version 57440 (0.0007) [2023-10-12 22:24:43,442][44958] Updated weights for policy 0, policy_version 57140 (0.0008) [2023-10-12 22:24:43,810][44958] Updated weights for policy 0, policy_version 57150 (0.0009) [2023-10-12 22:24:46,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117342208. Throughput: 0: 1624.5, 1: 1645.3. Samples: 29349206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:24:46,443][43579] Avg episode reward: [(0, '282.270'), (1, '287.160')] [2023-10-12 22:24:47,505][44959] Updated weights for policy 1, policy_version 57450 (0.0009) [2023-10-12 22:24:47,770][44958] Updated weights for policy 0, policy_version 57160 (0.0008) [2023-10-12 22:24:47,875][44959] Updated weights for policy 1, policy_version 57460 (0.0010) [2023-10-12 22:24:48,149][44958] Updated weights for policy 0, policy_version 57170 (0.0009) [2023-10-12 22:24:48,239][44959] Updated weights for policy 1, policy_version 57470 (0.0009) [2023-10-12 22:24:48,516][44958] Updated weights for policy 0, policy_version 57180 (0.0009) [2023-10-12 22:24:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117407744. Throughput: 0: 1629.6, 1: 1647.3. Samples: 29358096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:24:51,443][43579] Avg episode reward: [(0, '282.740'), (1, '288.020')] [2023-10-12 22:24:52,399][44959] Updated weights for policy 1, policy_version 57480 (0.0009) [2023-10-12 22:24:52,778][44959] Updated weights for policy 1, policy_version 57490 (0.0010) [2023-10-12 22:24:52,790][44958] Updated weights for policy 0, policy_version 57190 (0.0008) [2023-10-12 22:24:53,134][44959] Updated weights for policy 1, policy_version 57500 (0.0009) [2023-10-12 22:24:53,166][44958] Updated weights for policy 0, policy_version 57200 (0.0007) [2023-10-12 22:24:53,530][44958] Updated weights for policy 0, policy_version 57210 (0.0009) [2023-10-12 22:24:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117473280. Throughput: 0: 1640.8, 1: 1638.4. Samples: 29378464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:24:56,444][43579] Avg episode reward: [(0, '283.060'), (1, '288.940')] [2023-10-12 22:24:57,340][44959] Updated weights for policy 1, policy_version 57510 (0.0007) [2023-10-12 22:24:57,678][44958] Updated weights for policy 0, policy_version 57220 (0.0007) [2023-10-12 22:24:57,705][44959] Updated weights for policy 1, policy_version 57520 (0.0007) [2023-10-12 22:24:58,065][44958] Updated weights for policy 0, policy_version 57230 (0.0007) [2023-10-12 22:24:58,068][44959] Updated weights for policy 1, policy_version 57530 (0.0007) [2023-10-12 22:24:58,439][44958] Updated weights for policy 0, policy_version 57240 (0.0010) [2023-10-12 22:25:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117538816. Throughput: 0: 1627.1, 1: 1641.9. Samples: 29398290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:25:01,443][43579] Avg episode reward: [(0, '284.840'), (1, '286.930')] [2023-10-12 22:25:02,277][44959] Updated weights for policy 1, policy_version 57540 (0.0008) [2023-10-12 22:25:02,642][44959] Updated weights for policy 1, policy_version 57550 (0.0010) [2023-10-12 22:25:02,715][44958] Updated weights for policy 0, policy_version 57250 (0.0010) [2023-10-12 22:25:03,011][44959] Updated weights for policy 1, policy_version 57560 (0.0010) [2023-10-12 22:25:03,098][44958] Updated weights for policy 0, policy_version 57260 (0.0008) [2023-10-12 22:25:03,470][44958] Updated weights for policy 0, policy_version 57270 (0.0009) [2023-10-12 22:25:03,838][44958] Updated weights for policy 0, policy_version 57280 (0.0008) [2023-10-12 22:25:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117604352. Throughput: 0: 1627.3, 1: 1643.9. Samples: 29407038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:25:06,444][43579] Avg episode reward: [(0, '286.240'), (1, '284.180')] [2023-10-12 22:25:07,200][44959] Updated weights for policy 1, policy_version 57570 (0.0008) [2023-10-12 22:25:07,574][44959] Updated weights for policy 1, policy_version 57580 (0.0009) [2023-10-12 22:25:07,952][44959] Updated weights for policy 1, policy_version 57590 (0.0010) [2023-10-12 22:25:07,979][44958] Updated weights for policy 0, policy_version 57290 (0.0007) [2023-10-12 22:25:08,314][44959] Updated weights for policy 1, policy_version 57600 (0.0007) [2023-10-12 22:25:08,349][44958] Updated weights for policy 0, policy_version 57300 (0.0008) [2023-10-12 22:25:08,731][44958] Updated weights for policy 0, policy_version 57310 (0.0011) [2023-10-12 22:25:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117669888. Throughput: 0: 1627.4, 1: 1637.4. Samples: 29426904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:25:11,444][43579] Avg episode reward: [(0, '286.160'), (1, '284.630')] [2023-10-12 22:25:12,600][44959] Updated weights for policy 1, policy_version 57610 (0.0009) [2023-10-12 22:25:12,960][44958] Updated weights for policy 0, policy_version 57320 (0.0008) [2023-10-12 22:25:12,965][44959] Updated weights for policy 1, policy_version 57620 (0.0008) [2023-10-12 22:25:13,323][44958] Updated weights for policy 0, policy_version 57330 (0.0008) [2023-10-12 22:25:13,333][44959] Updated weights for policy 1, policy_version 57630 (0.0008) [2023-10-12 22:25:13,699][44958] Updated weights for policy 0, policy_version 57340 (0.0008) [2023-10-12 22:25:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117735424. Throughput: 0: 1626.6, 1: 1641.9. Samples: 29447462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:25:16,444][43579] Avg episode reward: [(0, '289.540'), (1, '280.200')] [2023-10-12 22:25:16,457][44518] Saving new best policy, reward=289.540! [2023-10-12 22:25:17,352][44959] Updated weights for policy 1, policy_version 57640 (0.0009) [2023-10-12 22:25:17,731][44959] Updated weights for policy 1, policy_version 57650 (0.0009) [2023-10-12 22:25:17,988][44958] Updated weights for policy 0, policy_version 57350 (0.0009) [2023-10-12 22:25:18,102][44959] Updated weights for policy 1, policy_version 57660 (0.0009) [2023-10-12 22:25:18,348][44958] Updated weights for policy 0, policy_version 57360 (0.0007) [2023-10-12 22:25:18,734][44958] Updated weights for policy 0, policy_version 57370 (0.0009) [2023-10-12 22:25:21,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117800960. Throughput: 0: 1626.5, 1: 1639.9. Samples: 29456202. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-12 22:25:21,443][43579] Avg episode reward: [(0, '288.900'), (1, '277.130')] [2023-10-12 22:25:22,395][44959] Updated weights for policy 1, policy_version 57670 (0.0008) [2023-10-12 22:25:22,726][44958] Updated weights for policy 0, policy_version 57380 (0.0011) [2023-10-12 22:25:22,768][44959] Updated weights for policy 1, policy_version 57680 (0.0007) [2023-10-12 22:25:23,097][44958] Updated weights for policy 0, policy_version 57390 (0.0008) [2023-10-12 22:25:23,135][44959] Updated weights for policy 1, policy_version 57690 (0.0010) [2023-10-12 22:25:23,470][44958] Updated weights for policy 0, policy_version 57400 (0.0008) [2023-10-12 22:25:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117866496. Throughput: 0: 1637.8, 1: 1634.1. Samples: 29476674. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-12 22:25:26,444][43579] Avg episode reward: [(0, '289.230'), (1, '279.990')] [2023-10-12 22:25:27,434][44959] Updated weights for policy 1, policy_version 57700 (0.0008) [2023-10-12 22:25:27,576][44958] Updated weights for policy 0, policy_version 57410 (0.0008) [2023-10-12 22:25:27,802][44959] Updated weights for policy 1, policy_version 57710 (0.0008) [2023-10-12 22:25:27,940][44958] Updated weights for policy 0, policy_version 57420 (0.0007) [2023-10-12 22:25:28,166][44959] Updated weights for policy 1, policy_version 57720 (0.0009) [2023-10-12 22:25:28,321][44958] Updated weights for policy 0, policy_version 57430 (0.0007) [2023-10-12 22:25:28,684][44958] Updated weights for policy 0, policy_version 57440 (0.0008) [2023-10-12 22:25:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117932032. Throughput: 0: 1634.5, 1: 1636.5. Samples: 29496402. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-12 22:25:31,443][43579] Avg episode reward: [(0, '290.500'), (1, '279.000')] [2023-10-12 22:25:31,452][44518] Saving new best policy, reward=290.500! [2023-10-12 22:25:32,323][44959] Updated weights for policy 1, policy_version 57730 (0.0009) [2023-10-12 22:25:32,700][44959] Updated weights for policy 1, policy_version 57740 (0.0009) [2023-10-12 22:25:33,050][44958] Updated weights for policy 0, policy_version 57450 (0.0008) [2023-10-12 22:25:33,064][44959] Updated weights for policy 1, policy_version 57750 (0.0009) [2023-10-12 22:25:33,419][44958] Updated weights for policy 0, policy_version 57460 (0.0008) [2023-10-12 22:25:33,434][44959] Updated weights for policy 1, policy_version 57760 (0.0007) [2023-10-12 22:25:33,799][44958] Updated weights for policy 0, policy_version 57470 (0.0009) [2023-10-12 22:25:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 117997568. Throughput: 0: 1634.7, 1: 1632.2. Samples: 29505110. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-12 22:25:36,444][43579] Avg episode reward: [(0, '288.360'), (1, '279.350')] [2023-10-12 22:25:37,598][44959] Updated weights for policy 1, policy_version 57770 (0.0010) [2023-10-12 22:25:37,812][44958] Updated weights for policy 0, policy_version 57480 (0.0009) [2023-10-12 22:25:37,968][44959] Updated weights for policy 1, policy_version 57780 (0.0009) [2023-10-12 22:25:38,182][44958] Updated weights for policy 0, policy_version 57490 (0.0008) [2023-10-12 22:25:38,337][44959] Updated weights for policy 1, policy_version 57790 (0.0008) [2023-10-12 22:25:38,554][44958] Updated weights for policy 0, policy_version 57500 (0.0011) [2023-10-12 22:25:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118063104. Throughput: 0: 1631.3, 1: 1631.4. Samples: 29525288. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-12 22:25:41,443][43579] Avg episode reward: [(0, '283.980'), (1, '279.060')] [2023-10-12 22:25:42,608][44959] Updated weights for policy 1, policy_version 57800 (0.0008) [2023-10-12 22:25:42,857][44958] Updated weights for policy 0, policy_version 57510 (0.0010) [2023-10-12 22:25:42,968][44959] Updated weights for policy 1, policy_version 57810 (0.0007) [2023-10-12 22:25:43,233][44958] Updated weights for policy 0, policy_version 57520 (0.0008) [2023-10-12 22:25:43,334][44959] Updated weights for policy 1, policy_version 57820 (0.0007) [2023-10-12 22:25:43,610][44958] Updated weights for policy 0, policy_version 57530 (0.0008) [2023-10-12 22:25:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118128640. Throughput: 0: 1638.5, 1: 1634.1. Samples: 29545558. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-12 22:25:46,443][43579] Avg episode reward: [(0, '280.590'), (1, '282.390')] [2023-10-12 22:25:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000057824_59211776.pth... [2023-10-12 22:25:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000057536_58916864.pth... [2023-10-12 22:25:46,490][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000056000_57344000.pth [2023-10-12 22:25:46,493][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000056288_57638912.pth [2023-10-12 22:25:47,424][44959] Updated weights for policy 1, policy_version 57830 (0.0008) [2023-10-12 22:25:47,607][44958] Updated weights for policy 0, policy_version 57540 (0.0007) [2023-10-12 22:25:47,793][44959] Updated weights for policy 1, policy_version 57840 (0.0008) [2023-10-12 22:25:47,975][44958] Updated weights for policy 0, policy_version 57550 (0.0010) [2023-10-12 22:25:48,158][44959] Updated weights for policy 1, policy_version 57850 (0.0009) [2023-10-12 22:25:48,348][44958] Updated weights for policy 0, policy_version 57560 (0.0010) [2023-10-12 22:25:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118194176. Throughput: 0: 1645.9, 1: 1631.8. Samples: 29554532. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-12 22:25:51,444][43579] Avg episode reward: [(0, '278.280'), (1, '284.950')] [2023-10-12 22:25:52,417][44958] Updated weights for policy 0, policy_version 57570 (0.0009) [2023-10-12 22:25:52,470][44959] Updated weights for policy 1, policy_version 57860 (0.0008) [2023-10-12 22:25:52,794][44958] Updated weights for policy 0, policy_version 57580 (0.0008) [2023-10-12 22:25:52,835][44959] Updated weights for policy 1, policy_version 57870 (0.0008) [2023-10-12 22:25:53,156][44958] Updated weights for policy 0, policy_version 57590 (0.0007) [2023-10-12 22:25:53,205][44959] Updated weights for policy 1, policy_version 57880 (0.0007) [2023-10-12 22:25:53,519][44958] Updated weights for policy 0, policy_version 57600 (0.0008) [2023-10-12 22:25:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118259712. Throughput: 0: 1649.2, 1: 1638.5. Samples: 29574852. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-12 22:25:56,444][43579] Avg episode reward: [(0, '275.210'), (1, '283.400')] [2023-10-12 22:25:57,367][44959] Updated weights for policy 1, policy_version 57890 (0.0007) [2023-10-12 22:25:57,728][44959] Updated weights for policy 1, policy_version 57900 (0.0007) [2023-10-12 22:25:57,848][44958] Updated weights for policy 0, policy_version 57610 (0.0008) [2023-10-12 22:25:58,093][44959] Updated weights for policy 1, policy_version 57910 (0.0008) [2023-10-12 22:25:58,219][44958] Updated weights for policy 0, policy_version 57620 (0.0008) [2023-10-12 22:25:58,466][44959] Updated weights for policy 1, policy_version 57920 (0.0008) [2023-10-12 22:25:58,590][44958] Updated weights for policy 0, policy_version 57630 (0.0009) [2023-10-12 22:26:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118325248. Throughput: 0: 1643.1, 1: 1634.9. Samples: 29594972. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 22:26:01,443][43579] Avg episode reward: [(0, '275.440'), (1, '284.210')] [2023-10-12 22:26:02,597][44959] Updated weights for policy 1, policy_version 57930 (0.0010) [2023-10-12 22:26:02,657][44958] Updated weights for policy 0, policy_version 57640 (0.0008) [2023-10-12 22:26:02,960][44959] Updated weights for policy 1, policy_version 57940 (0.0009) [2023-10-12 22:26:03,024][44958] Updated weights for policy 0, policy_version 57650 (0.0007) [2023-10-12 22:26:03,327][44959] Updated weights for policy 1, policy_version 57950 (0.0008) [2023-10-12 22:26:03,392][44958] Updated weights for policy 0, policy_version 57660 (0.0009) [2023-10-12 22:26:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118390784. Throughput: 0: 1648.2, 1: 1633.7. Samples: 29603886. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 22:26:06,444][43579] Avg episode reward: [(0, '278.810'), (1, '281.250')] [2023-10-12 22:26:07,443][44959] Updated weights for policy 1, policy_version 57960 (0.0007) [2023-10-12 22:26:07,688][44958] Updated weights for policy 0, policy_version 57670 (0.0008) [2023-10-12 22:26:07,806][44959] Updated weights for policy 1, policy_version 57970 (0.0008) [2023-10-12 22:26:08,062][44958] Updated weights for policy 0, policy_version 57680 (0.0007) [2023-10-12 22:26:08,178][44959] Updated weights for policy 1, policy_version 57980 (0.0008) [2023-10-12 22:26:08,437][44958] Updated weights for policy 0, policy_version 57690 (0.0010) [2023-10-12 22:26:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118456320. Throughput: 0: 1634.1, 1: 1644.7. Samples: 29624218. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 22:26:11,443][43579] Avg episode reward: [(0, '279.240'), (1, '280.960')] [2023-10-12 22:26:12,192][44959] Updated weights for policy 1, policy_version 57990 (0.0009) [2023-10-12 22:26:12,565][44959] Updated weights for policy 1, policy_version 58000 (0.0008) [2023-10-12 22:26:12,742][44958] Updated weights for policy 0, policy_version 57700 (0.0010) [2023-10-12 22:26:12,942][44959] Updated weights for policy 1, policy_version 58010 (0.0007) [2023-10-12 22:26:13,109][44958] Updated weights for policy 0, policy_version 57710 (0.0008) [2023-10-12 22:26:13,497][44958] Updated weights for policy 0, policy_version 57720 (0.0009) [2023-10-12 22:26:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118521856. Throughput: 0: 1639.6, 1: 1651.5. Samples: 29644504. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 22:26:16,444][43579] Avg episode reward: [(0, '281.480'), (1, '281.160')] [2023-10-12 22:26:16,968][44959] Updated weights for policy 1, policy_version 58020 (0.0008) [2023-10-12 22:26:17,340][44959] Updated weights for policy 1, policy_version 58030 (0.0011) [2023-10-12 22:26:17,548][44958] Updated weights for policy 0, policy_version 57730 (0.0009) [2023-10-12 22:26:17,709][44959] Updated weights for policy 1, policy_version 58040 (0.0007) [2023-10-12 22:26:17,909][44958] Updated weights for policy 0, policy_version 57740 (0.0008) [2023-10-12 22:26:18,282][44958] Updated weights for policy 0, policy_version 57750 (0.0010) [2023-10-12 22:26:18,659][44958] Updated weights for policy 0, policy_version 57760 (0.0008) [2023-10-12 22:26:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118587392. Throughput: 0: 1638.9, 1: 1657.3. Samples: 29653438. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 22:26:21,443][43579] Avg episode reward: [(0, '277.610'), (1, '275.330')] [2023-10-12 22:26:21,932][44959] Updated weights for policy 1, policy_version 58050 (0.0010) [2023-10-12 22:26:22,313][44959] Updated weights for policy 1, policy_version 58060 (0.0011) [2023-10-12 22:26:22,678][44959] Updated weights for policy 1, policy_version 58070 (0.0008) [2023-10-12 22:26:22,938][44958] Updated weights for policy 0, policy_version 57770 (0.0009) [2023-10-12 22:26:23,052][44959] Updated weights for policy 1, policy_version 58080 (0.0007) [2023-10-12 22:26:23,310][44958] Updated weights for policy 0, policy_version 57780 (0.0008) [2023-10-12 22:26:23,687][44958] Updated weights for policy 0, policy_version 57790 (0.0011) [2023-10-12 22:26:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118652928. Throughput: 0: 1635.8, 1: 1657.7. Samples: 29673496. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 22:26:26,443][43579] Avg episode reward: [(0, '277.780'), (1, '272.630')] [2023-10-12 22:26:27,118][44959] Updated weights for policy 1, policy_version 58090 (0.0008) [2023-10-12 22:26:27,492][44959] Updated weights for policy 1, policy_version 58100 (0.0007) [2023-10-12 22:26:27,854][44959] Updated weights for policy 1, policy_version 58110 (0.0008) [2023-10-12 22:26:27,909][44958] Updated weights for policy 0, policy_version 57800 (0.0009) [2023-10-12 22:26:28,281][44958] Updated weights for policy 0, policy_version 57810 (0.0011) [2023-10-12 22:26:28,662][44958] Updated weights for policy 0, policy_version 57820 (0.0007) [2023-10-12 22:26:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118718464. Throughput: 0: 1636.3, 1: 1658.2. Samples: 29693810. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 22:26:31,443][43579] Avg episode reward: [(0, '276.520'), (1, '273.510')] [2023-10-12 22:26:31,958][44959] Updated weights for policy 1, policy_version 58120 (0.0007) [2023-10-12 22:26:32,325][44959] Updated weights for policy 1, policy_version 58130 (0.0009) [2023-10-12 22:26:32,691][44959] Updated weights for policy 1, policy_version 58140 (0.0009) [2023-10-12 22:26:32,834][44958] Updated weights for policy 0, policy_version 57830 (0.0010) [2023-10-12 22:26:33,211][44958] Updated weights for policy 0, policy_version 57840 (0.0007) [2023-10-12 22:26:33,584][44958] Updated weights for policy 0, policy_version 57850 (0.0011) [2023-10-12 22:26:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118784000. Throughput: 0: 1629.1, 1: 1663.4. Samples: 29702692. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-12 22:26:36,443][43579] Avg episode reward: [(0, '275.380'), (1, '276.640')] [2023-10-12 22:26:36,769][44959] Updated weights for policy 1, policy_version 58150 (0.0008) [2023-10-12 22:26:37,127][44959] Updated weights for policy 1, policy_version 58160 (0.0007) [2023-10-12 22:26:37,494][44959] Updated weights for policy 1, policy_version 58170 (0.0008) [2023-10-12 22:26:37,702][44958] Updated weights for policy 0, policy_version 57860 (0.0008) [2023-10-12 22:26:38,069][44958] Updated weights for policy 0, policy_version 57870 (0.0007) [2023-10-12 22:26:38,448][44958] Updated weights for policy 0, policy_version 57880 (0.0009) [2023-10-12 22:26:41,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 118849536. Throughput: 0: 1626.9, 1: 1666.3. Samples: 29723046. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) [2023-10-12 22:26:41,444][43579] Avg episode reward: [(0, '276.080'), (1, '276.700')] [2023-10-12 22:26:41,571][44959] Updated weights for policy 1, policy_version 58180 (0.0008) [2023-10-12 22:26:41,935][44959] Updated weights for policy 1, policy_version 58190 (0.0007) [2023-10-12 22:26:42,306][44959] Updated weights for policy 1, policy_version 58200 (0.0008) [2023-10-12 22:26:42,789][44958] Updated weights for policy 0, policy_version 57890 (0.0009) [2023-10-12 22:26:43,171][44958] Updated weights for policy 0, policy_version 57900 (0.0007) [2023-10-12 22:26:43,546][44958] Updated weights for policy 0, policy_version 57910 (0.0009) [2023-10-12 22:26:43,916][44958] Updated weights for policy 0, policy_version 57920 (0.0007) [2023-10-12 22:26:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118915072. Throughput: 0: 1640.4, 1: 1669.7. Samples: 29743928. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) [2023-10-12 22:26:46,444][43579] Avg episode reward: [(0, '275.690'), (1, '280.630')] [2023-10-12 22:26:46,496][44959] Updated weights for policy 1, policy_version 58210 (0.0008) [2023-10-12 22:26:46,868][44959] Updated weights for policy 1, policy_version 58220 (0.0008) [2023-10-12 22:26:47,235][44959] Updated weights for policy 1, policy_version 58230 (0.0010) [2023-10-12 22:26:47,603][44959] Updated weights for policy 1, policy_version 58240 (0.0009) [2023-10-12 22:26:47,835][44958] Updated weights for policy 0, policy_version 57930 (0.0008) [2023-10-12 22:26:48,203][44958] Updated weights for policy 0, policy_version 57940 (0.0010) [2023-10-12 22:26:48,575][44958] Updated weights for policy 0, policy_version 57950 (0.0010) [2023-10-12 22:26:51,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 118980608. Throughput: 0: 1636.0, 1: 1669.1. Samples: 29752614. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) [2023-10-12 22:26:51,443][43579] Avg episode reward: [(0, '277.330'), (1, '284.610')] [2023-10-12 22:26:51,852][44959] Updated weights for policy 1, policy_version 58250 (0.0008) [2023-10-12 22:26:52,217][44959] Updated weights for policy 1, policy_version 58260 (0.0008) [2023-10-12 22:26:52,587][44959] Updated weights for policy 1, policy_version 58270 (0.0007) [2023-10-12 22:26:52,699][44958] Updated weights for policy 0, policy_version 57960 (0.0008) [2023-10-12 22:26:53,067][44958] Updated weights for policy 0, policy_version 57970 (0.0007) [2023-10-12 22:26:53,452][44958] Updated weights for policy 0, policy_version 57980 (0.0010) [2023-10-12 22:26:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119046144. Throughput: 0: 1647.8, 1: 1659.5. Samples: 29773048. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) [2023-10-12 22:26:56,444][43579] Avg episode reward: [(0, '274.200'), (1, '285.180')] [2023-10-12 22:26:56,642][44959] Updated weights for policy 1, policy_version 58280 (0.0008) [2023-10-12 22:26:57,008][44959] Updated weights for policy 1, policy_version 58290 (0.0011) [2023-10-12 22:26:57,384][44959] Updated weights for policy 1, policy_version 58300 (0.0009) [2023-10-12 22:26:57,618][44958] Updated weights for policy 0, policy_version 57990 (0.0009) [2023-10-12 22:26:57,988][44958] Updated weights for policy 0, policy_version 58000 (0.0008) [2023-10-12 22:26:58,354][44958] Updated weights for policy 0, policy_version 58010 (0.0008) [2023-10-12 22:27:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119111680. Throughput: 0: 1652.0, 1: 1662.8. Samples: 29793668. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) [2023-10-12 22:27:01,443][43579] Avg episode reward: [(0, '277.460'), (1, '284.220')] [2023-10-12 22:27:01,568][44959] Updated weights for policy 1, policy_version 58310 (0.0008) [2023-10-12 22:27:01,952][44959] Updated weights for policy 1, policy_version 58320 (0.0008) [2023-10-12 22:27:02,328][44959] Updated weights for policy 1, policy_version 58330 (0.0008) [2023-10-12 22:27:02,560][44958] Updated weights for policy 0, policy_version 58020 (0.0009) [2023-10-12 22:27:02,938][44958] Updated weights for policy 0, policy_version 58030 (0.0009) [2023-10-12 22:27:03,303][44958] Updated weights for policy 0, policy_version 58040 (0.0009) [2023-10-12 22:27:06,415][44959] Updated weights for policy 1, policy_version 58340 (0.0008) [2023-10-12 22:27:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119177216. Throughput: 0: 1651.2, 1: 1657.1. Samples: 29802312. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) [2023-10-12 22:27:06,443][43579] Avg episode reward: [(0, '278.460'), (1, '281.730')] [2023-10-12 22:27:06,783][44959] Updated weights for policy 1, policy_version 58350 (0.0008) [2023-10-12 22:27:07,155][44959] Updated weights for policy 1, policy_version 58360 (0.0007) [2023-10-12 22:27:07,451][44958] Updated weights for policy 0, policy_version 58050 (0.0008) [2023-10-12 22:27:07,825][44958] Updated weights for policy 0, policy_version 58060 (0.0010) [2023-10-12 22:27:08,199][44958] Updated weights for policy 0, policy_version 58070 (0.0008) [2023-10-12 22:27:08,569][44958] Updated weights for policy 0, policy_version 58080 (0.0009) [2023-10-12 22:27:11,365][44959] Updated weights for policy 1, policy_version 58370 (0.0007) [2023-10-12 22:27:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119242752. Throughput: 0: 1650.7, 1: 1662.0. Samples: 29822564. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) [2023-10-12 22:27:11,443][43579] Avg episode reward: [(0, '277.630'), (1, '278.260')] [2023-10-12 22:27:11,737][44959] Updated weights for policy 1, policy_version 58380 (0.0010) [2023-10-12 22:27:12,117][44959] Updated weights for policy 1, policy_version 58390 (0.0011) [2023-10-12 22:27:12,480][44959] Updated weights for policy 1, policy_version 58400 (0.0008) [2023-10-12 22:27:12,595][44958] Updated weights for policy 0, policy_version 58090 (0.0007) [2023-10-12 22:27:12,966][44958] Updated weights for policy 0, policy_version 58100 (0.0007) [2023-10-12 22:27:13,336][44958] Updated weights for policy 0, policy_version 58110 (0.0009) [2023-10-12 22:27:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 119308288. Throughput: 0: 1654.9, 1: 1661.0. Samples: 29843024. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) [2023-10-12 22:27:16,443][43579] Avg episode reward: [(0, '277.790'), (1, '272.400')] [2023-10-12 22:27:16,628][44959] Updated weights for policy 1, policy_version 58410 (0.0007) [2023-10-12 22:27:16,997][44959] Updated weights for policy 1, policy_version 58420 (0.0007) [2023-10-12 22:27:17,365][44959] Updated weights for policy 1, policy_version 58430 (0.0008) [2023-10-12 22:27:17,622][44958] Updated weights for policy 0, policy_version 58120 (0.0010) [2023-10-12 22:27:18,008][44958] Updated weights for policy 0, policy_version 58130 (0.0009) [2023-10-12 22:27:18,383][44958] Updated weights for policy 0, policy_version 58140 (0.0010) [2023-10-12 22:27:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119373824. Throughput: 0: 1653.2, 1: 1658.6. Samples: 29851720. Policy #0 lag: (min: 16.0, avg: 38.6, max: 48.0) [2023-10-12 22:27:21,443][43579] Avg episode reward: [(0, '277.570'), (1, '276.590')] [2023-10-12 22:27:21,517][44959] Updated weights for policy 1, policy_version 58440 (0.0009) [2023-10-12 22:27:21,890][44959] Updated weights for policy 1, policy_version 58450 (0.0007) [2023-10-12 22:27:22,253][44959] Updated weights for policy 1, policy_version 58460 (0.0008) [2023-10-12 22:27:22,521][44958] Updated weights for policy 0, policy_version 58150 (0.0009) [2023-10-12 22:27:22,888][44958] Updated weights for policy 0, policy_version 58160 (0.0009) [2023-10-12 22:27:23,267][44958] Updated weights for policy 0, policy_version 58170 (0.0007) [2023-10-12 22:27:26,366][44959] Updated weights for policy 1, policy_version 58470 (0.0008) [2023-10-12 22:27:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119439360. Throughput: 0: 1658.6, 1: 1653.5. Samples: 29872090. Policy #0 lag: (min: 16.0, avg: 38.6, max: 48.0) [2023-10-12 22:27:26,443][43579] Avg episode reward: [(0, '280.990'), (1, '279.690')] [2023-10-12 22:27:26,730][44959] Updated weights for policy 1, policy_version 58480 (0.0010) [2023-10-12 22:27:27,099][44959] Updated weights for policy 1, policy_version 58490 (0.0011) [2023-10-12 22:27:27,330][44958] Updated weights for policy 0, policy_version 58180 (0.0010) [2023-10-12 22:27:27,707][44958] Updated weights for policy 0, policy_version 58190 (0.0009) [2023-10-12 22:27:28,078][44958] Updated weights for policy 0, policy_version 58200 (0.0008) [2023-10-12 22:27:31,256][44959] Updated weights for policy 1, policy_version 58500 (0.0007) [2023-10-12 22:27:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119504896. Throughput: 0: 1648.9, 1: 1648.7. Samples: 29892320. Policy #0 lag: (min: 16.0, avg: 38.6, max: 48.0) [2023-10-12 22:27:31,443][43579] Avg episode reward: [(0, '279.140'), (1, '274.300')] [2023-10-12 22:27:31,628][44959] Updated weights for policy 1, policy_version 58510 (0.0008) [2023-10-12 22:27:31,996][44959] Updated weights for policy 1, policy_version 58520 (0.0008) [2023-10-12 22:27:32,352][44958] Updated weights for policy 0, policy_version 58210 (0.0008) [2023-10-12 22:27:32,724][44958] Updated weights for policy 0, policy_version 58220 (0.0008) [2023-10-12 22:27:33,097][44958] Updated weights for policy 0, policy_version 58230 (0.0007) [2023-10-12 22:27:33,476][44958] Updated weights for policy 0, policy_version 58240 (0.0010) [2023-10-12 22:27:36,238][44959] Updated weights for policy 1, policy_version 58530 (0.0008) [2023-10-12 22:27:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119570432. Throughput: 0: 1649.3, 1: 1652.6. Samples: 29901200. Policy #0 lag: (min: 16.0, avg: 38.6, max: 48.0) [2023-10-12 22:27:36,444][43579] Avg episode reward: [(0, '276.970'), (1, '273.300')] [2023-10-12 22:27:36,601][44959] Updated weights for policy 1, policy_version 58540 (0.0010) [2023-10-12 22:27:36,981][44959] Updated weights for policy 1, policy_version 58550 (0.0011) [2023-10-12 22:27:37,340][44959] Updated weights for policy 1, policy_version 58560 (0.0009) [2023-10-12 22:27:37,573][44958] Updated weights for policy 0, policy_version 58250 (0.0008) [2023-10-12 22:27:37,937][44958] Updated weights for policy 0, policy_version 58260 (0.0009) [2023-10-12 22:27:38,319][44958] Updated weights for policy 0, policy_version 58270 (0.0008) [2023-10-12 22:27:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119635968. Throughput: 0: 1645.7, 1: 1657.0. Samples: 29921670. Policy #0 lag: (min: 16.0, avg: 38.6, max: 48.0) [2023-10-12 22:27:41,444][43579] Avg episode reward: [(0, '272.470'), (1, '274.620')] [2023-10-12 22:27:41,468][44959] Updated weights for policy 1, policy_version 58570 (0.0009) [2023-10-12 22:27:41,848][44959] Updated weights for policy 1, policy_version 58580 (0.0009) [2023-10-12 22:27:42,205][44959] Updated weights for policy 1, policy_version 58590 (0.0009) [2023-10-12 22:27:42,362][44958] Updated weights for policy 0, policy_version 58280 (0.0008) [2023-10-12 22:27:42,737][44958] Updated weights for policy 0, policy_version 58290 (0.0008) [2023-10-12 22:27:43,096][44958] Updated weights for policy 0, policy_version 58300 (0.0011) [2023-10-12 22:27:46,237][44959] Updated weights for policy 1, policy_version 58600 (0.0008) [2023-10-12 22:27:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119701504. Throughput: 0: 1647.5, 1: 1651.4. Samples: 29942118. Policy #0 lag: (min: 16.0, avg: 38.6, max: 48.0) [2023-10-12 22:27:46,443][43579] Avg episode reward: [(0, '273.790'), (1, '270.880')] [2023-10-12 22:27:46,450][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000058304_59703296.pth... [2023-10-12 22:27:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000056768_58130432.pth [2023-10-12 22:27:46,611][44959] Updated weights for policy 1, policy_version 58610 (0.0009) [2023-10-12 22:27:46,974][44959] Updated weights for policy 1, policy_version 58620 (0.0009) [2023-10-12 22:27:47,110][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000058624_60030976.pth... [2023-10-12 22:27:47,139][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000057056_58425344.pth [2023-10-12 22:27:47,269][44958] Updated weights for policy 0, policy_version 58310 (0.0008) [2023-10-12 22:27:47,637][44958] Updated weights for policy 0, policy_version 58320 (0.0010) [2023-10-12 22:27:48,007][44958] Updated weights for policy 0, policy_version 58330 (0.0008) [2023-10-12 22:27:51,244][44959] Updated weights for policy 1, policy_version 58630 (0.0007) [2023-10-12 22:27:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119767040. Throughput: 0: 1648.9, 1: 1655.2. Samples: 29950998. Policy #0 lag: (min: 16.0, avg: 38.6, max: 48.0) [2023-10-12 22:27:51,443][43579] Avg episode reward: [(0, '271.770'), (1, '271.120')] [2023-10-12 22:27:51,611][44959] Updated weights for policy 1, policy_version 58640 (0.0010) [2023-10-12 22:27:51,976][44959] Updated weights for policy 1, policy_version 58650 (0.0008) [2023-10-12 22:27:52,310][44958] Updated weights for policy 0, policy_version 58340 (0.0010) [2023-10-12 22:27:52,680][44958] Updated weights for policy 0, policy_version 58350 (0.0008) [2023-10-12 22:27:53,046][44958] Updated weights for policy 0, policy_version 58360 (0.0010) [2023-10-12 22:27:56,104][44959] Updated weights for policy 1, policy_version 58660 (0.0010) [2023-10-12 22:27:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119832576. Throughput: 0: 1647.5, 1: 1653.9. Samples: 29971130. Policy #0 lag: (min: 16.0, avg: 38.6, max: 48.0) [2023-10-12 22:27:56,444][43579] Avg episode reward: [(0, '272.410'), (1, '270.520')] [2023-10-12 22:27:56,478][44959] Updated weights for policy 1, policy_version 58670 (0.0009) [2023-10-12 22:27:56,856][44959] Updated weights for policy 1, policy_version 58680 (0.0007) [2023-10-12 22:27:57,187][44958] Updated weights for policy 0, policy_version 58370 (0.0009) [2023-10-12 22:27:57,553][44958] Updated weights for policy 0, policy_version 58380 (0.0008) [2023-10-12 22:27:57,923][44958] Updated weights for policy 0, policy_version 58390 (0.0008) [2023-10-12 22:27:58,292][44958] Updated weights for policy 0, policy_version 58400 (0.0009) [2023-10-12 22:28:01,028][44959] Updated weights for policy 1, policy_version 58690 (0.0009) [2023-10-12 22:28:01,388][44959] Updated weights for policy 1, policy_version 58700 (0.0007) [2023-10-12 22:28:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 119898112. Throughput: 0: 1645.3, 1: 1647.9. Samples: 29991222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:28:01,444][43579] Avg episode reward: [(0, '272.520'), (1, '280.660')] [2023-10-12 22:28:01,754][44959] Updated weights for policy 1, policy_version 58710 (0.0008) [2023-10-12 22:28:02,118][44959] Updated weights for policy 1, policy_version 58720 (0.0007) [2023-10-12 22:28:02,460][44958] Updated weights for policy 0, policy_version 58410 (0.0011) [2023-10-12 22:28:02,828][44958] Updated weights for policy 0, policy_version 58420 (0.0008) [2023-10-12 22:28:03,200][44958] Updated weights for policy 0, policy_version 58430 (0.0007) [2023-10-12 22:28:06,273][44959] Updated weights for policy 1, policy_version 58730 (0.0009) [2023-10-12 22:28:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 119963648. Throughput: 0: 1647.9, 1: 1650.9. Samples: 30000166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:28:06,443][43579] Avg episode reward: [(0, '272.530'), (1, '280.980')] [2023-10-12 22:28:06,643][44959] Updated weights for policy 1, policy_version 58740 (0.0008) [2023-10-12 22:28:07,013][44959] Updated weights for policy 1, policy_version 58750 (0.0010) [2023-10-12 22:28:07,401][44958] Updated weights for policy 0, policy_version 58440 (0.0008) [2023-10-12 22:28:07,784][44958] Updated weights for policy 0, policy_version 58450 (0.0009) [2023-10-12 22:28:08,161][44958] Updated weights for policy 0, policy_version 58460 (0.0010) [2023-10-12 22:28:11,064][44959] Updated weights for policy 1, policy_version 58760 (0.0008) [2023-10-12 22:28:11,421][44959] Updated weights for policy 1, policy_version 58770 (0.0010) [2023-10-12 22:28:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120029184. Throughput: 0: 1646.5, 1: 1657.3. Samples: 30020764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:28:11,443][43579] Avg episode reward: [(0, '278.420'), (1, '285.290')] [2023-10-12 22:28:11,785][44959] Updated weights for policy 1, policy_version 58780 (0.0009) [2023-10-12 22:28:12,221][44958] Updated weights for policy 0, policy_version 58470 (0.0009) [2023-10-12 22:28:12,593][44958] Updated weights for policy 0, policy_version 58480 (0.0009) [2023-10-12 22:28:12,969][44958] Updated weights for policy 0, policy_version 58490 (0.0009) [2023-10-12 22:28:15,955][44959] Updated weights for policy 1, policy_version 58790 (0.0007) [2023-10-12 22:28:16,313][44959] Updated weights for policy 1, policy_version 58800 (0.0008) [2023-10-12 22:28:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120094720. Throughput: 0: 1649.3, 1: 1650.4. Samples: 30040804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:28:16,443][43579] Avg episode reward: [(0, '281.650'), (1, '288.150')] [2023-10-12 22:28:16,677][44959] Updated weights for policy 1, policy_version 58810 (0.0009) [2023-10-12 22:28:17,032][44958] Updated weights for policy 0, policy_version 58500 (0.0010) [2023-10-12 22:28:17,400][44958] Updated weights for policy 0, policy_version 58510 (0.0007) [2023-10-12 22:28:17,774][44958] Updated weights for policy 0, policy_version 58520 (0.0007) [2023-10-12 22:28:20,941][44959] Updated weights for policy 1, policy_version 58820 (0.0010) [2023-10-12 22:28:21,304][44959] Updated weights for policy 1, policy_version 58830 (0.0009) [2023-10-12 22:28:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120160256. Throughput: 0: 1651.2, 1: 1654.4. Samples: 30049950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:28:21,444][43579] Avg episode reward: [(0, '285.810'), (1, '286.340')] [2023-10-12 22:28:21,667][44959] Updated weights for policy 1, policy_version 58840 (0.0008) [2023-10-12 22:28:21,946][44958] Updated weights for policy 0, policy_version 58530 (0.0007) [2023-10-12 22:28:22,310][44958] Updated weights for policy 0, policy_version 58540 (0.0009) [2023-10-12 22:28:22,679][44958] Updated weights for policy 0, policy_version 58550 (0.0007) [2023-10-12 22:28:23,052][44958] Updated weights for policy 0, policy_version 58560 (0.0008) [2023-10-12 22:28:25,879][44959] Updated weights for policy 1, policy_version 58850 (0.0008) [2023-10-12 22:28:26,239][44959] Updated weights for policy 1, policy_version 58860 (0.0009) [2023-10-12 22:28:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120225792. Throughput: 0: 1650.4, 1: 1654.3. Samples: 30070380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:28:26,443][43579] Avg episode reward: [(0, '280.830'), (1, '286.850')] [2023-10-12 22:28:26,613][44959] Updated weights for policy 1, policy_version 58870 (0.0008) [2023-10-12 22:28:26,972][44959] Updated weights for policy 1, policy_version 58880 (0.0009) [2023-10-12 22:28:27,131][44958] Updated weights for policy 0, policy_version 58570 (0.0008) [2023-10-12 22:28:27,503][44958] Updated weights for policy 0, policy_version 58580 (0.0010) [2023-10-12 22:28:27,876][44958] Updated weights for policy 0, policy_version 58590 (0.0008) [2023-10-12 22:28:31,208][44959] Updated weights for policy 1, policy_version 58890 (0.0009) [2023-10-12 22:28:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120291328. Throughput: 0: 1649.6, 1: 1645.3. Samples: 30090388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:28:31,443][43579] Avg episode reward: [(0, '281.890'), (1, '283.130')] [2023-10-12 22:28:31,574][44959] Updated weights for policy 1, policy_version 58900 (0.0009) [2023-10-12 22:28:31,901][44958] Updated weights for policy 0, policy_version 58600 (0.0008) [2023-10-12 22:28:31,946][44959] Updated weights for policy 1, policy_version 58910 (0.0008) [2023-10-12 22:28:32,264][44958] Updated weights for policy 0, policy_version 58610 (0.0007) [2023-10-12 22:28:32,639][44958] Updated weights for policy 0, policy_version 58620 (0.0008) [2023-10-12 22:28:36,260][44959] Updated weights for policy 1, policy_version 58920 (0.0008) [2023-10-12 22:28:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120356864. Throughput: 0: 1651.9, 1: 1647.5. Samples: 30099472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:28:36,443][43579] Avg episode reward: [(0, '284.100'), (1, '270.320')] [2023-10-12 22:28:36,636][44959] Updated weights for policy 1, policy_version 58930 (0.0009) [2023-10-12 22:28:36,746][44958] Updated weights for policy 0, policy_version 58630 (0.0008) [2023-10-12 22:28:37,000][44959] Updated weights for policy 1, policy_version 58940 (0.0009) [2023-10-12 22:28:37,114][44958] Updated weights for policy 0, policy_version 58640 (0.0007) [2023-10-12 22:28:37,492][44958] Updated weights for policy 0, policy_version 58650 (0.0007) [2023-10-12 22:28:40,977][44959] Updated weights for policy 1, policy_version 58950 (0.0009) [2023-10-12 22:28:41,349][44959] Updated weights for policy 1, policy_version 58960 (0.0008) [2023-10-12 22:28:41,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120422400. Throughput: 0: 1659.3, 1: 1646.8. Samples: 30119906. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 22:28:41,444][43579] Avg episode reward: [(0, '283.950'), (1, '270.180')] [2023-10-12 22:28:41,592][44958] Updated weights for policy 0, policy_version 58660 (0.0010) [2023-10-12 22:28:41,713][44959] Updated weights for policy 1, policy_version 58970 (0.0009) [2023-10-12 22:28:41,965][44958] Updated weights for policy 0, policy_version 58670 (0.0009) [2023-10-12 22:28:42,335][44958] Updated weights for policy 0, policy_version 58680 (0.0010) [2023-10-12 22:28:45,967][44959] Updated weights for policy 1, policy_version 58980 (0.0008) [2023-10-12 22:28:46,334][44959] Updated weights for policy 1, policy_version 58990 (0.0007) [2023-10-12 22:28:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120487936. Throughput: 0: 1668.0, 1: 1646.0. Samples: 30140352. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 22:28:46,443][43579] Avg episode reward: [(0, '281.710'), (1, '275.490')] [2023-10-12 22:28:46,566][44958] Updated weights for policy 0, policy_version 58690 (0.0009) [2023-10-12 22:28:46,698][44959] Updated weights for policy 1, policy_version 59000 (0.0007) [2023-10-12 22:28:46,975][44958] Updated weights for policy 0, policy_version 58700 (0.0008) [2023-10-12 22:28:47,348][44958] Updated weights for policy 0, policy_version 58710 (0.0007) [2023-10-12 22:28:47,710][44958] Updated weights for policy 0, policy_version 58720 (0.0008) [2023-10-12 22:28:50,874][44959] Updated weights for policy 1, policy_version 59010 (0.0008) [2023-10-12 22:28:51,241][44959] Updated weights for policy 1, policy_version 59020 (0.0010) [2023-10-12 22:28:51,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120553472. Throughput: 0: 1663.8, 1: 1648.1. Samples: 30149200. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 22:28:51,443][43579] Avg episode reward: [(0, '284.530'), (1, '272.120')] [2023-10-12 22:28:51,602][44959] Updated weights for policy 1, policy_version 59030 (0.0009) [2023-10-12 22:28:51,810][44958] Updated weights for policy 0, policy_version 58730 (0.0007) [2023-10-12 22:28:51,973][44959] Updated weights for policy 1, policy_version 59040 (0.0010) [2023-10-12 22:28:52,179][44958] Updated weights for policy 0, policy_version 58740 (0.0008) [2023-10-12 22:28:52,559][44958] Updated weights for policy 0, policy_version 58750 (0.0008) [2023-10-12 22:28:55,957][44959] Updated weights for policy 1, policy_version 59050 (0.0010) [2023-10-12 22:28:56,326][44959] Updated weights for policy 1, policy_version 59060 (0.0008) [2023-10-12 22:28:56,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120619008. Throughput: 0: 1663.9, 1: 1644.5. Samples: 30169642. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 22:28:56,444][43579] Avg episode reward: [(0, '286.470'), (1, '278.010')] [2023-10-12 22:28:56,645][44958] Updated weights for policy 0, policy_version 58760 (0.0008) [2023-10-12 22:28:56,694][44959] Updated weights for policy 1, policy_version 59070 (0.0007) [2023-10-12 22:28:57,021][44958] Updated weights for policy 0, policy_version 58770 (0.0009) [2023-10-12 22:28:57,392][44958] Updated weights for policy 0, policy_version 58780 (0.0008) [2023-10-12 22:29:00,875][44959] Updated weights for policy 1, policy_version 59080 (0.0009) [2023-10-12 22:29:01,244][44959] Updated weights for policy 1, policy_version 59090 (0.0008) [2023-10-12 22:29:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120684544. Throughput: 0: 1661.0, 1: 1641.5. Samples: 30189414. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 22:29:01,444][43579] Avg episode reward: [(0, '282.190'), (1, '280.250')] [2023-10-12 22:29:01,509][44958] Updated weights for policy 0, policy_version 58790 (0.0008) [2023-10-12 22:29:01,620][44959] Updated weights for policy 1, policy_version 59100 (0.0008) [2023-10-12 22:29:01,879][44958] Updated weights for policy 0, policy_version 58800 (0.0008) [2023-10-12 22:29:02,256][44958] Updated weights for policy 0, policy_version 58810 (0.0008) [2023-10-12 22:29:05,676][44959] Updated weights for policy 1, policy_version 59110 (0.0008) [2023-10-12 22:29:06,036][44959] Updated weights for policy 1, policy_version 59120 (0.0009) [2023-10-12 22:29:06,247][44958] Updated weights for policy 0, policy_version 58820 (0.0009) [2023-10-12 22:29:06,407][44959] Updated weights for policy 1, policy_version 59130 (0.0010) [2023-10-12 22:29:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120750080. Throughput: 0: 1661.8, 1: 1649.2. Samples: 30198944. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 22:29:06,443][43579] Avg episode reward: [(0, '280.230'), (1, '289.050')] [2023-10-12 22:29:06,616][44958] Updated weights for policy 0, policy_version 58830 (0.0009) [2023-10-12 22:29:06,985][44958] Updated weights for policy 0, policy_version 58840 (0.0009) [2023-10-12 22:29:10,718][44959] Updated weights for policy 1, policy_version 59140 (0.0008) [2023-10-12 22:29:11,089][44959] Updated weights for policy 1, policy_version 59150 (0.0008) [2023-10-12 22:29:11,251][44958] Updated weights for policy 0, policy_version 58850 (0.0009) [2023-10-12 22:29:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120815616. Throughput: 0: 1656.3, 1: 1648.0. Samples: 30219074. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 22:29:11,444][43579] Avg episode reward: [(0, '282.370'), (1, '286.560')] [2023-10-12 22:29:11,461][44959] Updated weights for policy 1, policy_version 59160 (0.0008) [2023-10-12 22:29:11,628][44958] Updated weights for policy 0, policy_version 58860 (0.0010) [2023-10-12 22:29:12,001][44958] Updated weights for policy 0, policy_version 58870 (0.0010) [2023-10-12 22:29:12,377][44958] Updated weights for policy 0, policy_version 58880 (0.0008) [2023-10-12 22:29:15,763][44959] Updated weights for policy 1, policy_version 59170 (0.0008) [2023-10-12 22:29:16,140][44959] Updated weights for policy 1, policy_version 59180 (0.0008) [2023-10-12 22:29:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120881152. Throughput: 0: 1654.0, 1: 1645.4. Samples: 30238862. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) [2023-10-12 22:29:16,443][43579] Avg episode reward: [(0, '282.310'), (1, '285.360')] [2023-10-12 22:29:16,513][44959] Updated weights for policy 1, policy_version 59190 (0.0008) [2023-10-12 22:29:16,730][44958] Updated weights for policy 0, policy_version 58890 (0.0009) [2023-10-12 22:29:16,873][44959] Updated weights for policy 1, policy_version 59200 (0.0008) [2023-10-12 22:29:17,113][44958] Updated weights for policy 0, policy_version 58900 (0.0009) [2023-10-12 22:29:17,492][44958] Updated weights for policy 0, policy_version 58910 (0.0010) [2023-10-12 22:29:21,143][44959] Updated weights for policy 1, policy_version 59210 (0.0007) [2023-10-12 22:29:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 120946688. Throughput: 0: 1651.3, 1: 1654.6. Samples: 30248240. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:29:21,444][43579] Avg episode reward: [(0, '280.860'), (1, '288.850')] [2023-10-12 22:29:21,520][44959] Updated weights for policy 1, policy_version 59220 (0.0007) [2023-10-12 22:29:21,567][44958] Updated weights for policy 0, policy_version 58920 (0.0010) [2023-10-12 22:29:21,890][44959] Updated weights for policy 1, policy_version 59230 (0.0008) [2023-10-12 22:29:21,943][44958] Updated weights for policy 0, policy_version 58930 (0.0007) [2023-10-12 22:29:22,308][44958] Updated weights for policy 0, policy_version 58940 (0.0010) [2023-10-12 22:29:25,842][44959] Updated weights for policy 1, policy_version 59240 (0.0007) [2023-10-12 22:29:26,213][44959] Updated weights for policy 1, policy_version 59250 (0.0009) [2023-10-12 22:29:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 121012224. Throughput: 0: 1654.2, 1: 1648.5. Samples: 30268526. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:29:26,443][43579] Avg episode reward: [(0, '273.380'), (1, '287.380')] [2023-10-12 22:29:26,491][44958] Updated weights for policy 0, policy_version 58950 (0.0009) [2023-10-12 22:29:26,578][44959] Updated weights for policy 1, policy_version 59260 (0.0009) [2023-10-12 22:29:26,862][44958] Updated weights for policy 0, policy_version 58960 (0.0008) [2023-10-12 22:29:27,238][44958] Updated weights for policy 0, policy_version 58970 (0.0008) [2023-10-12 22:29:30,696][44959] Updated weights for policy 1, policy_version 59270 (0.0007) [2023-10-12 22:29:31,070][44959] Updated weights for policy 1, policy_version 59280 (0.0008) [2023-10-12 22:29:31,178][44958] Updated weights for policy 0, policy_version 58980 (0.0008) [2023-10-12 22:29:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 121077760. Throughput: 0: 1647.0, 1: 1638.3. Samples: 30288190. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:29:31,443][43579] Avg episode reward: [(0, '277.160'), (1, '285.500')] [2023-10-12 22:29:31,443][44959] Updated weights for policy 1, policy_version 59290 (0.0007) [2023-10-12 22:29:31,550][44958] Updated weights for policy 0, policy_version 58990 (0.0009) [2023-10-12 22:29:31,918][44958] Updated weights for policy 0, policy_version 59000 (0.0009) [2023-10-12 22:29:35,584][44959] Updated weights for policy 1, policy_version 59300 (0.0007) [2023-10-12 22:29:35,953][44959] Updated weights for policy 1, policy_version 59310 (0.0009) [2023-10-12 22:29:36,124][44958] Updated weights for policy 0, policy_version 59010 (0.0009) [2023-10-12 22:29:36,321][44959] Updated weights for policy 1, policy_version 59320 (0.0008) [2023-10-12 22:29:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 121143296. Throughput: 0: 1655.0, 1: 1646.9. Samples: 30297786. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:29:36,443][43579] Avg episode reward: [(0, '280.000'), (1, '280.580')] [2023-10-12 22:29:36,534][44958] Updated weights for policy 0, policy_version 59020 (0.0008) [2023-10-12 22:29:36,899][44958] Updated weights for policy 0, policy_version 59030 (0.0010) [2023-10-12 22:29:37,273][44958] Updated weights for policy 0, policy_version 59040 (0.0010) [2023-10-12 22:29:40,445][44959] Updated weights for policy 1, policy_version 59330 (0.0010) [2023-10-12 22:29:40,814][44959] Updated weights for policy 1, policy_version 59340 (0.0009) [2023-10-12 22:29:41,178][44959] Updated weights for policy 1, policy_version 59350 (0.0009) [2023-10-12 22:29:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 121208832. Throughput: 0: 1644.2, 1: 1645.3. Samples: 30317666. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:29:41,443][43579] Avg episode reward: [(0, '279.420'), (1, '284.500')] [2023-10-12 22:29:41,541][44959] Updated weights for policy 1, policy_version 59360 (0.0007) [2023-10-12 22:29:41,613][44958] Updated weights for policy 0, policy_version 59050 (0.0009) [2023-10-12 22:29:41,992][44958] Updated weights for policy 0, policy_version 59060 (0.0007) [2023-10-12 22:29:42,358][44958] Updated weights for policy 0, policy_version 59070 (0.0010) [2023-10-12 22:29:45,578][44959] Updated weights for policy 1, policy_version 59370 (0.0009) [2023-10-12 22:29:45,951][44959] Updated weights for policy 1, policy_version 59380 (0.0008) [2023-10-12 22:29:46,321][44959] Updated weights for policy 1, policy_version 59390 (0.0008) [2023-10-12 22:29:46,443][43579] Fps is (10 sec: 16383.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 121307136. Throughput: 0: 1647.6, 1: 1640.2. Samples: 30337362. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:29:46,444][43579] Avg episode reward: [(0, '283.720'), (1, '283.350')] [2023-10-12 22:29:46,456][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000059392_60817408.pth... [2023-10-12 22:29:46,489][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000057824_59211776.pth [2023-10-12 22:29:46,514][44958] Updated weights for policy 0, policy_version 59080 (0.0008) [2023-10-12 22:29:46,896][44958] Updated weights for policy 0, policy_version 59090 (0.0008) [2023-10-12 22:29:47,276][44958] Updated weights for policy 0, policy_version 59100 (0.0010) [2023-10-12 22:29:47,418][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000059104_60522496.pth... [2023-10-12 22:29:47,458][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000057536_58916864.pth [2023-10-12 22:29:50,665][44959] Updated weights for policy 1, policy_version 59400 (0.0009) [2023-10-12 22:29:51,027][44959] Updated weights for policy 1, policy_version 59410 (0.0008) [2023-10-12 22:29:51,396][44959] Updated weights for policy 1, policy_version 59420 (0.0007) [2023-10-12 22:29:51,410][44958] Updated weights for policy 0, policy_version 59110 (0.0009) [2023-10-12 22:29:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 121339904. Throughput: 0: 1643.3, 1: 1643.7. Samples: 30346860. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:29:51,443][43579] Avg episode reward: [(0, '282.460'), (1, '283.860')] [2023-10-12 22:29:51,778][44958] Updated weights for policy 0, policy_version 59120 (0.0008) [2023-10-12 22:29:52,149][44958] Updated weights for policy 0, policy_version 59130 (0.0007) [2023-10-12 22:29:55,600][44959] Updated weights for policy 1, policy_version 59430 (0.0008) [2023-10-12 22:29:55,967][44959] Updated weights for policy 1, policy_version 59440 (0.0010) [2023-10-12 22:29:56,339][44959] Updated weights for policy 1, policy_version 59450 (0.0009) [2023-10-12 22:29:56,357][44958] Updated weights for policy 0, policy_version 59140 (0.0007) [2023-10-12 22:29:56,442][43579] Fps is (10 sec: 9830.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 121405440. Throughput: 0: 1643.8, 1: 1640.1. Samples: 30366852. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:29:56,443][43579] Avg episode reward: [(0, '289.000'), (1, '277.800')] [2023-10-12 22:29:56,725][44958] Updated weights for policy 0, policy_version 59150 (0.0007) [2023-10-12 22:29:57,096][44958] Updated weights for policy 0, policy_version 59160 (0.0008) [2023-10-12 22:30:00,261][44959] Updated weights for policy 1, policy_version 59460 (0.0008) [2023-10-12 22:30:00,621][44959] Updated weights for policy 1, policy_version 59470 (0.0010) [2023-10-12 22:30:00,986][44959] Updated weights for policy 1, policy_version 59480 (0.0010) [2023-10-12 22:30:01,260][44958] Updated weights for policy 0, policy_version 59170 (0.0008) [2023-10-12 22:30:01,442][43579] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 121503744. Throughput: 0: 1644.6, 1: 1641.2. Samples: 30386720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:30:01,443][43579] Avg episode reward: [(0, '283.800'), (1, '279.530')] [2023-10-12 22:30:01,633][44958] Updated weights for policy 0, policy_version 59180 (0.0008) [2023-10-12 22:30:02,006][44958] Updated weights for policy 0, policy_version 59190 (0.0009) [2023-10-12 22:30:02,375][44958] Updated weights for policy 0, policy_version 59200 (0.0009) [2023-10-12 22:30:05,183][44959] Updated weights for policy 1, policy_version 59490 (0.0008) [2023-10-12 22:30:05,599][44959] Updated weights for policy 1, policy_version 59500 (0.0007) [2023-10-12 22:30:05,977][44959] Updated weights for policy 1, policy_version 59510 (0.0007) [2023-10-12 22:30:06,348][44959] Updated weights for policy 1, policy_version 59520 (0.0008) [2023-10-12 22:30:06,443][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 121569280. Throughput: 0: 1640.0, 1: 1651.5. Samples: 30396354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:30:06,443][43579] Avg episode reward: [(0, '281.610'), (1, '282.790')] [2023-10-12 22:30:06,689][44958] Updated weights for policy 0, policy_version 59210 (0.0007) [2023-10-12 22:30:07,061][44958] Updated weights for policy 0, policy_version 59220 (0.0007) [2023-10-12 22:30:07,432][44958] Updated weights for policy 0, policy_version 59230 (0.0007) [2023-10-12 22:30:10,341][44959] Updated weights for policy 1, policy_version 59530 (0.0009) [2023-10-12 22:30:10,707][44959] Updated weights for policy 1, policy_version 59540 (0.0009) [2023-10-12 22:30:11,070][44959] Updated weights for policy 1, policy_version 59550 (0.0011) [2023-10-12 22:30:11,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 121634816. Throughput: 0: 1630.1, 1: 1653.9. Samples: 30416306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:30:11,443][43579] Avg episode reward: [(0, '283.460'), (1, '277.870')] [2023-10-12 22:30:11,650][44958] Updated weights for policy 0, policy_version 59240 (0.0009) [2023-10-12 22:30:12,017][44958] Updated weights for policy 0, policy_version 59250 (0.0008) [2023-10-12 22:30:12,397][44958] Updated weights for policy 0, policy_version 59260 (0.0007) [2023-10-12 22:30:15,324][44959] Updated weights for policy 1, policy_version 59560 (0.0008) [2023-10-12 22:30:15,683][44959] Updated weights for policy 1, policy_version 59570 (0.0010) [2023-10-12 22:30:16,050][44959] Updated weights for policy 1, policy_version 59580 (0.0008) [2023-10-12 22:30:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 121700352. Throughput: 0: 1634.1, 1: 1649.7. Samples: 30435962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:30:16,443][43579] Avg episode reward: [(0, '283.210'), (1, '278.610')] [2023-10-12 22:30:16,539][44958] Updated weights for policy 0, policy_version 59270 (0.0007) [2023-10-12 22:30:16,901][44958] Updated weights for policy 0, policy_version 59280 (0.0010) [2023-10-12 22:30:17,273][44958] Updated weights for policy 0, policy_version 59290 (0.0009) [2023-10-12 22:30:20,169][44959] Updated weights for policy 1, policy_version 59590 (0.0010) [2023-10-12 22:30:20,533][44959] Updated weights for policy 1, policy_version 59600 (0.0010) [2023-10-12 22:30:20,907][44959] Updated weights for policy 1, policy_version 59610 (0.0008) [2023-10-12 22:30:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 121765888. Throughput: 0: 1630.5, 1: 1659.0. Samples: 30445814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:30:21,444][43579] Avg episode reward: [(0, '281.060'), (1, '277.110')] [2023-10-12 22:30:21,654][44958] Updated weights for policy 0, policy_version 59300 (0.0008) [2023-10-12 22:30:22,026][44958] Updated weights for policy 0, policy_version 59310 (0.0010) [2023-10-12 22:30:22,399][44958] Updated weights for policy 0, policy_version 59320 (0.0007) [2023-10-12 22:30:25,386][44959] Updated weights for policy 1, policy_version 59620 (0.0007) [2023-10-12 22:30:25,758][44959] Updated weights for policy 1, policy_version 59630 (0.0008) [2023-10-12 22:30:26,120][44959] Updated weights for policy 1, policy_version 59640 (0.0008) [2023-10-12 22:30:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 121831424. Throughput: 0: 1635.4, 1: 1653.3. Samples: 30465656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:30:26,443][43579] Avg episode reward: [(0, '279.160'), (1, '283.100')] [2023-10-12 22:30:26,545][44958] Updated weights for policy 0, policy_version 59330 (0.0009) [2023-10-12 22:30:26,922][44958] Updated weights for policy 0, policy_version 59340 (0.0009) [2023-10-12 22:30:27,298][44958] Updated weights for policy 0, policy_version 59350 (0.0008) [2023-10-12 22:30:27,671][44958] Updated weights for policy 0, policy_version 59360 (0.0008) [2023-10-12 22:30:30,174][44959] Updated weights for policy 1, policy_version 59650 (0.0008) [2023-10-12 22:30:30,544][44959] Updated weights for policy 1, policy_version 59660 (0.0008) [2023-10-12 22:30:30,919][44959] Updated weights for policy 1, policy_version 59670 (0.0008) [2023-10-12 22:30:31,280][44959] Updated weights for policy 1, policy_version 59680 (0.0009) [2023-10-12 22:30:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 121896960. Throughput: 0: 1635.0, 1: 1650.1. Samples: 30485190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:30:31,443][43579] Avg episode reward: [(0, '284.000'), (1, '283.210')] [2023-10-12 22:30:31,709][44958] Updated weights for policy 0, policy_version 59370 (0.0008) [2023-10-12 22:30:32,086][44958] Updated weights for policy 0, policy_version 59380 (0.0009) [2023-10-12 22:30:32,456][44958] Updated weights for policy 0, policy_version 59390 (0.0010) [2023-10-12 22:30:35,383][44959] Updated weights for policy 1, policy_version 59690 (0.0010) [2023-10-12 22:30:35,746][44959] Updated weights for policy 1, policy_version 59700 (0.0010) [2023-10-12 22:30:36,124][44959] Updated weights for policy 1, policy_version 59710 (0.0008) [2023-10-12 22:30:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 121962496. Throughput: 0: 1634.4, 1: 1658.2. Samples: 30495030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:30:36,443][43579] Avg episode reward: [(0, '276.560'), (1, '281.660')] [2023-10-12 22:30:36,707][44958] Updated weights for policy 0, policy_version 59400 (0.0009) [2023-10-12 22:30:37,079][44958] Updated weights for policy 0, policy_version 59410 (0.0010) [2023-10-12 22:30:37,452][44958] Updated weights for policy 0, policy_version 59420 (0.0007) [2023-10-12 22:30:40,328][44959] Updated weights for policy 1, policy_version 59720 (0.0009) [2023-10-12 22:30:40,705][44959] Updated weights for policy 1, policy_version 59730 (0.0009) [2023-10-12 22:30:41,075][44959] Updated weights for policy 1, policy_version 59740 (0.0008) [2023-10-12 22:30:41,423][44958] Updated weights for policy 0, policy_version 59430 (0.0008) [2023-10-12 22:30:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 122028032. Throughput: 0: 1641.2, 1: 1657.2. Samples: 30515280. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-12 22:30:41,443][43579] Avg episode reward: [(0, '270.770'), (1, '283.080')] [2023-10-12 22:30:41,789][44958] Updated weights for policy 0, policy_version 59440 (0.0009) [2023-10-12 22:30:42,164][44958] Updated weights for policy 0, policy_version 59450 (0.0010) [2023-10-12 22:30:45,051][44959] Updated weights for policy 1, policy_version 59750 (0.0007) [2023-10-12 22:30:45,426][44959] Updated weights for policy 1, policy_version 59760 (0.0008) [2023-10-12 22:30:45,793][44959] Updated weights for policy 1, policy_version 59770 (0.0009) [2023-10-12 22:30:46,409][44958] Updated weights for policy 0, policy_version 59460 (0.0010) [2023-10-12 22:30:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122093568. Throughput: 0: 1638.2, 1: 1644.9. Samples: 30534460. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-12 22:30:46,444][43579] Avg episode reward: [(0, '269.920'), (1, '285.320')] [2023-10-12 22:30:46,779][44958] Updated weights for policy 0, policy_version 59470 (0.0009) [2023-10-12 22:30:47,153][44958] Updated weights for policy 0, policy_version 59480 (0.0008) [2023-10-12 22:30:50,139][44959] Updated weights for policy 1, policy_version 59780 (0.0009) [2023-10-12 22:30:50,540][44959] Updated weights for policy 1, policy_version 59790 (0.0010) [2023-10-12 22:30:50,916][44959] Updated weights for policy 1, policy_version 59800 (0.0011) [2023-10-12 22:30:51,403][44958] Updated weights for policy 0, policy_version 59490 (0.0008) [2023-10-12 22:30:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 122159104. Throughput: 0: 1641.6, 1: 1647.4. Samples: 30544360. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-12 22:30:51,444][43579] Avg episode reward: [(0, '270.080'), (1, '286.000')] [2023-10-12 22:30:51,788][44958] Updated weights for policy 0, policy_version 59500 (0.0009) [2023-10-12 22:30:52,160][44958] Updated weights for policy 0, policy_version 59510 (0.0008) [2023-10-12 22:30:52,534][44958] Updated weights for policy 0, policy_version 59520 (0.0009) [2023-10-12 22:30:54,929][44959] Updated weights for policy 1, policy_version 59810 (0.0009) [2023-10-12 22:30:55,301][44959] Updated weights for policy 1, policy_version 59820 (0.0008) [2023-10-12 22:30:55,669][44959] Updated weights for policy 1, policy_version 59830 (0.0007) [2023-10-12 22:30:56,040][44959] Updated weights for policy 1, policy_version 59840 (0.0008) [2023-10-12 22:30:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 122224640. Throughput: 0: 1648.4, 1: 1640.8. Samples: 30564324. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-12 22:30:56,443][43579] Avg episode reward: [(0, '269.350'), (1, '279.550')] [2023-10-12 22:30:56,583][44958] Updated weights for policy 0, policy_version 59530 (0.0007) [2023-10-12 22:30:56,952][44958] Updated weights for policy 0, policy_version 59540 (0.0010) [2023-10-12 22:30:57,320][44958] Updated weights for policy 0, policy_version 59550 (0.0009) [2023-10-12 22:31:00,499][44959] Updated weights for policy 1, policy_version 59850 (0.0009) [2023-10-12 22:31:00,868][44959] Updated weights for policy 1, policy_version 59860 (0.0009) [2023-10-12 22:31:01,226][44959] Updated weights for policy 1, policy_version 59870 (0.0008) [2023-10-12 22:31:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122290176. Throughput: 0: 1644.7, 1: 1644.8. Samples: 30583990. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-12 22:31:01,443][43579] Avg episode reward: [(0, '272.780'), (1, '278.700')] [2023-10-12 22:31:01,452][44958] Updated weights for policy 0, policy_version 59560 (0.0007) [2023-10-12 22:31:01,818][44958] Updated weights for policy 0, policy_version 59570 (0.0009) [2023-10-12 22:31:02,182][44958] Updated weights for policy 0, policy_version 59580 (0.0008) [2023-10-12 22:31:05,184][44959] Updated weights for policy 1, policy_version 59880 (0.0009) [2023-10-12 22:31:05,546][44959] Updated weights for policy 1, policy_version 59890 (0.0011) [2023-10-12 22:31:05,922][44959] Updated weights for policy 1, policy_version 59900 (0.0011) [2023-10-12 22:31:06,364][44958] Updated weights for policy 0, policy_version 59590 (0.0011) [2023-10-12 22:31:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122355712. Throughput: 0: 1646.9, 1: 1644.3. Samples: 30593920. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-12 22:31:06,443][43579] Avg episode reward: [(0, '272.990'), (1, '285.130')] [2023-10-12 22:31:06,738][44958] Updated weights for policy 0, policy_version 59600 (0.0011) [2023-10-12 22:31:07,111][44958] Updated weights for policy 0, policy_version 59610 (0.0010) [2023-10-12 22:31:10,021][44959] Updated weights for policy 1, policy_version 59910 (0.0010) [2023-10-12 22:31:10,388][44959] Updated weights for policy 1, policy_version 59920 (0.0009) [2023-10-12 22:31:10,756][44959] Updated weights for policy 1, policy_version 59930 (0.0007) [2023-10-12 22:31:11,216][44958] Updated weights for policy 0, policy_version 59620 (0.0008) [2023-10-12 22:31:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122421248. Throughput: 0: 1651.8, 1: 1649.9. Samples: 30614232. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-12 22:31:11,444][43579] Avg episode reward: [(0, '278.290'), (1, '271.130')] [2023-10-12 22:31:11,583][44958] Updated weights for policy 0, policy_version 59630 (0.0007) [2023-10-12 22:31:11,953][44958] Updated weights for policy 0, policy_version 59640 (0.0008) [2023-10-12 22:31:14,862][44959] Updated weights for policy 1, policy_version 59940 (0.0008) [2023-10-12 22:31:15,225][44959] Updated weights for policy 1, policy_version 59950 (0.0007) [2023-10-12 22:31:15,597][44959] Updated weights for policy 1, policy_version 59960 (0.0008) [2023-10-12 22:31:16,166][44958] Updated weights for policy 0, policy_version 59650 (0.0010) [2023-10-12 22:31:16,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 122486784. Throughput: 0: 1642.9, 1: 1650.2. Samples: 30633380. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-12 22:31:16,444][43579] Avg episode reward: [(0, '281.410'), (1, '268.080')] [2023-10-12 22:31:16,534][44958] Updated weights for policy 0, policy_version 59660 (0.0010) [2023-10-12 22:31:16,906][44958] Updated weights for policy 0, policy_version 59670 (0.0009) [2023-10-12 22:31:17,270][44958] Updated weights for policy 0, policy_version 59680 (0.0010) [2023-10-12 22:31:19,721][44959] Updated weights for policy 1, policy_version 59970 (0.0008) [2023-10-12 22:31:20,078][44959] Updated weights for policy 1, policy_version 59980 (0.0008) [2023-10-12 22:31:20,461][44959] Updated weights for policy 1, policy_version 59990 (0.0009) [2023-10-12 22:31:20,829][44959] Updated weights for policy 1, policy_version 60000 (0.0009) [2023-10-12 22:31:21,261][44958] Updated weights for policy 0, policy_version 59690 (0.0010) [2023-10-12 22:31:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122552320. Throughput: 0: 1649.1, 1: 1652.7. Samples: 30643610. Policy #0 lag: (min: 19.0, avg: 21.5, max: 51.0) [2023-10-12 22:31:21,443][43579] Avg episode reward: [(0, '281.040'), (1, '271.240')] [2023-10-12 22:31:21,624][44958] Updated weights for policy 0, policy_version 59700 (0.0010) [2023-10-12 22:31:21,997][44958] Updated weights for policy 0, policy_version 59710 (0.0007) [2023-10-12 22:31:25,160][44959] Updated weights for policy 1, policy_version 60010 (0.0007) [2023-10-12 22:31:25,517][44959] Updated weights for policy 1, policy_version 60020 (0.0007) [2023-10-12 22:31:25,888][44959] Updated weights for policy 1, policy_version 60030 (0.0009) [2023-10-12 22:31:26,227][44958] Updated weights for policy 0, policy_version 59720 (0.0008) [2023-10-12 22:31:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122617856. Throughput: 0: 1650.1, 1: 1647.7. Samples: 30663682. Policy #0 lag: (min: 19.0, avg: 21.5, max: 51.0) [2023-10-12 22:31:26,443][43579] Avg episode reward: [(0, '285.980'), (1, '269.160')] [2023-10-12 22:31:26,607][44958] Updated weights for policy 0, policy_version 59730 (0.0009) [2023-10-12 22:31:26,963][44958] Updated weights for policy 0, policy_version 59740 (0.0008) [2023-10-12 22:31:29,752][44959] Updated weights for policy 1, policy_version 60040 (0.0008) [2023-10-12 22:31:30,130][44959] Updated weights for policy 1, policy_version 60050 (0.0008) [2023-10-12 22:31:30,499][44959] Updated weights for policy 1, policy_version 60060 (0.0008) [2023-10-12 22:31:31,395][44958] Updated weights for policy 0, policy_version 59750 (0.0008) [2023-10-12 22:31:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122683392. Throughput: 0: 1646.0, 1: 1651.9. Samples: 30682862. Policy #0 lag: (min: 19.0, avg: 21.5, max: 51.0) [2023-10-12 22:31:31,444][43579] Avg episode reward: [(0, '286.780'), (1, '266.620')] [2023-10-12 22:31:31,766][44958] Updated weights for policy 0, policy_version 59760 (0.0007) [2023-10-12 22:31:32,155][44958] Updated weights for policy 0, policy_version 59770 (0.0010) [2023-10-12 22:31:34,652][44959] Updated weights for policy 1, policy_version 60070 (0.0008) [2023-10-12 22:31:35,017][44959] Updated weights for policy 1, policy_version 60080 (0.0010) [2023-10-12 22:31:35,387][44959] Updated weights for policy 1, policy_version 60090 (0.0008) [2023-10-12 22:31:36,124][44958] Updated weights for policy 0, policy_version 59780 (0.0007) [2023-10-12 22:31:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122748928. Throughput: 0: 1650.0, 1: 1656.3. Samples: 30693146. Policy #0 lag: (min: 19.0, avg: 21.5, max: 51.0) [2023-10-12 22:31:36,443][43579] Avg episode reward: [(0, '287.390'), (1, '267.890')] [2023-10-12 22:31:36,507][44958] Updated weights for policy 0, policy_version 59790 (0.0007) [2023-10-12 22:31:36,876][44958] Updated weights for policy 0, policy_version 59800 (0.0007) [2023-10-12 22:31:39,537][44959] Updated weights for policy 1, policy_version 60100 (0.0009) [2023-10-12 22:31:39,939][44959] Updated weights for policy 1, policy_version 60110 (0.0010) [2023-10-12 22:31:40,311][44959] Updated weights for policy 1, policy_version 60120 (0.0008) [2023-10-12 22:31:40,933][44958] Updated weights for policy 0, policy_version 59810 (0.0008) [2023-10-12 22:31:41,292][44958] Updated weights for policy 0, policy_version 59820 (0.0008) [2023-10-12 22:31:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122814464. Throughput: 0: 1650.4, 1: 1649.0. Samples: 30712796. Policy #0 lag: (min: 19.0, avg: 21.5, max: 51.0) [2023-10-12 22:31:41,443][43579] Avg episode reward: [(0, '285.960'), (1, '274.720')] [2023-10-12 22:31:41,662][44958] Updated weights for policy 0, policy_version 59830 (0.0009) [2023-10-12 22:31:42,035][44958] Updated weights for policy 0, policy_version 59840 (0.0008) [2023-10-12 22:31:44,304][44959] Updated weights for policy 1, policy_version 60130 (0.0008) [2023-10-12 22:31:44,682][44959] Updated weights for policy 1, policy_version 60140 (0.0007) [2023-10-12 22:31:45,048][44959] Updated weights for policy 1, policy_version 60150 (0.0008) [2023-10-12 22:31:45,419][44959] Updated weights for policy 1, policy_version 60160 (0.0009) [2023-10-12 22:31:46,291][44958] Updated weights for policy 0, policy_version 59850 (0.0010) [2023-10-12 22:31:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122880000. Throughput: 0: 1638.1, 1: 1656.6. Samples: 30732252. Policy #0 lag: (min: 19.0, avg: 21.5, max: 51.0) [2023-10-12 22:31:46,444][43579] Avg episode reward: [(0, '279.480'), (1, '276.100')] [2023-10-12 22:31:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000060160_61603840.pth... [2023-10-12 22:31:46,488][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000058624_60030976.pth [2023-10-12 22:31:46,653][44958] Updated weights for policy 0, policy_version 59860 (0.0009) [2023-10-12 22:31:47,035][44958] Updated weights for policy 0, policy_version 59870 (0.0010) [2023-10-12 22:31:47,099][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000059872_61308928.pth... [2023-10-12 22:31:47,127][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000058304_59703296.pth [2023-10-12 22:31:49,802][44959] Updated weights for policy 1, policy_version 60170 (0.0009) [2023-10-12 22:31:50,159][44959] Updated weights for policy 1, policy_version 60180 (0.0007) [2023-10-12 22:31:50,531][44959] Updated weights for policy 1, policy_version 60190 (0.0007) [2023-10-12 22:31:51,317][44958] Updated weights for policy 0, policy_version 59880 (0.0008) [2023-10-12 22:31:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 122945536. Throughput: 0: 1639.6, 1: 1660.3. Samples: 30742416. Policy #0 lag: (min: 19.0, avg: 21.5, max: 51.0) [2023-10-12 22:31:51,443][43579] Avg episode reward: [(0, '275.870'), (1, '283.020')] [2023-10-12 22:31:51,696][44958] Updated weights for policy 0, policy_version 59890 (0.0008) [2023-10-12 22:31:52,081][44958] Updated weights for policy 0, policy_version 59900 (0.0009) [2023-10-12 22:31:54,475][44959] Updated weights for policy 1, policy_version 60200 (0.0008) [2023-10-12 22:31:54,842][44959] Updated weights for policy 1, policy_version 60210 (0.0008) [2023-10-12 22:31:55,218][44959] Updated weights for policy 1, policy_version 60220 (0.0011) [2023-10-12 22:31:56,288][44958] Updated weights for policy 0, policy_version 59910 (0.0008) [2023-10-12 22:31:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123011072. Throughput: 0: 1639.1, 1: 1642.1. Samples: 30761886. Policy #0 lag: (min: 19.0, avg: 21.5, max: 51.0) [2023-10-12 22:31:56,444][43579] Avg episode reward: [(0, '271.480'), (1, '278.240')] [2023-10-12 22:31:56,658][44958] Updated weights for policy 0, policy_version 59920 (0.0011) [2023-10-12 22:31:57,021][44958] Updated weights for policy 0, policy_version 59930 (0.0011) [2023-10-12 22:31:59,428][44959] Updated weights for policy 1, policy_version 60230 (0.0008) [2023-10-12 22:31:59,793][44959] Updated weights for policy 1, policy_version 60240 (0.0009) [2023-10-12 22:32:00,155][44959] Updated weights for policy 1, policy_version 60250 (0.0009) [2023-10-12 22:32:01,032][44958] Updated weights for policy 0, policy_version 59940 (0.0009) [2023-10-12 22:32:01,412][44958] Updated weights for policy 0, policy_version 59950 (0.0007) [2023-10-12 22:32:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123076608. Throughput: 0: 1638.6, 1: 1651.6. Samples: 30781442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:01,444][43579] Avg episode reward: [(0, '273.080'), (1, '272.820')] [2023-10-12 22:32:01,785][44958] Updated weights for policy 0, policy_version 59960 (0.0008) [2023-10-12 22:32:04,239][44959] Updated weights for policy 1, policy_version 60260 (0.0010) [2023-10-12 22:32:04,612][44959] Updated weights for policy 1, policy_version 60270 (0.0009) [2023-10-12 22:32:04,976][44959] Updated weights for policy 1, policy_version 60280 (0.0010) [2023-10-12 22:32:05,927][44958] Updated weights for policy 0, policy_version 59970 (0.0010) [2023-10-12 22:32:06,303][44958] Updated weights for policy 0, policy_version 59980 (0.0010) [2023-10-12 22:32:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123142144. Throughput: 0: 1645.5, 1: 1654.3. Samples: 30792100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:06,444][43579] Avg episode reward: [(0, '274.090'), (1, '277.350')] [2023-10-12 22:32:06,665][44958] Updated weights for policy 0, policy_version 59990 (0.0009) [2023-10-12 22:32:07,045][44958] Updated weights for policy 0, policy_version 60000 (0.0011) [2023-10-12 22:32:09,173][44959] Updated weights for policy 1, policy_version 60290 (0.0008) [2023-10-12 22:32:09,541][44959] Updated weights for policy 1, policy_version 60300 (0.0009) [2023-10-12 22:32:09,915][44959] Updated weights for policy 1, policy_version 60310 (0.0011) [2023-10-12 22:32:10,284][44959] Updated weights for policy 1, policy_version 60320 (0.0010) [2023-10-12 22:32:11,292][44958] Updated weights for policy 0, policy_version 60010 (0.0008) [2023-10-12 22:32:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123207680. Throughput: 0: 1641.2, 1: 1643.1. Samples: 30811472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:11,443][43579] Avg episode reward: [(0, '276.200'), (1, '278.300')] [2023-10-12 22:32:11,664][44958] Updated weights for policy 0, policy_version 60020 (0.0008) [2023-10-12 22:32:12,033][44958] Updated weights for policy 0, policy_version 60030 (0.0010) [2023-10-12 22:32:14,663][44959] Updated weights for policy 1, policy_version 60330 (0.0008) [2023-10-12 22:32:15,027][44959] Updated weights for policy 1, policy_version 60340 (0.0007) [2023-10-12 22:32:15,403][44959] Updated weights for policy 1, policy_version 60350 (0.0007) [2023-10-12 22:32:16,110][44958] Updated weights for policy 0, policy_version 60040 (0.0008) [2023-10-12 22:32:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123273216. Throughput: 0: 1642.0, 1: 1652.9. Samples: 30831136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:16,444][43579] Avg episode reward: [(0, '277.130'), (1, '276.480')] [2023-10-12 22:32:16,491][44958] Updated weights for policy 0, policy_version 60050 (0.0008) [2023-10-12 22:32:16,851][44958] Updated weights for policy 0, policy_version 60060 (0.0008) [2023-10-12 22:32:19,468][44959] Updated weights for policy 1, policy_version 60360 (0.0008) [2023-10-12 22:32:19,831][44959] Updated weights for policy 1, policy_version 60370 (0.0008) [2023-10-12 22:32:20,198][44959] Updated weights for policy 1, policy_version 60380 (0.0008) [2023-10-12 22:32:21,023][44958] Updated weights for policy 0, policy_version 60070 (0.0008) [2023-10-12 22:32:21,396][44958] Updated weights for policy 0, policy_version 60080 (0.0007) [2023-10-12 22:32:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123338752. Throughput: 0: 1646.3, 1: 1652.1. Samples: 30841572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:21,443][43579] Avg episode reward: [(0, '278.770'), (1, '275.580')] [2023-10-12 22:32:21,775][44958] Updated weights for policy 0, policy_version 60090 (0.0007) [2023-10-12 22:32:24,344][44959] Updated weights for policy 1, policy_version 60390 (0.0007) [2023-10-12 22:32:24,713][44959] Updated weights for policy 1, policy_version 60400 (0.0010) [2023-10-12 22:32:25,087][44959] Updated weights for policy 1, policy_version 60410 (0.0007) [2023-10-12 22:32:26,167][44958] Updated weights for policy 0, policy_version 60100 (0.0009) [2023-10-12 22:32:26,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123404288. Throughput: 0: 1641.1, 1: 1646.4. Samples: 30860734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:26,443][43579] Avg episode reward: [(0, '279.240'), (1, '281.660')] [2023-10-12 22:32:26,527][44958] Updated weights for policy 0, policy_version 60110 (0.0008) [2023-10-12 22:32:26,902][44958] Updated weights for policy 0, policy_version 60120 (0.0009) [2023-10-12 22:32:29,161][44959] Updated weights for policy 1, policy_version 60420 (0.0007) [2023-10-12 22:32:29,524][44959] Updated weights for policy 1, policy_version 60430 (0.0008) [2023-10-12 22:32:29,896][44959] Updated weights for policy 1, policy_version 60440 (0.0008) [2023-10-12 22:32:31,078][44958] Updated weights for policy 0, policy_version 60130 (0.0008) [2023-10-12 22:32:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123469824. Throughput: 0: 1647.6, 1: 1651.4. Samples: 30880704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:31,443][43579] Avg episode reward: [(0, '278.560'), (1, '285.690')] [2023-10-12 22:32:31,464][44958] Updated weights for policy 0, policy_version 60140 (0.0008) [2023-10-12 22:32:31,825][44958] Updated weights for policy 0, policy_version 60150 (0.0007) [2023-10-12 22:32:32,194][44958] Updated weights for policy 0, policy_version 60160 (0.0011) [2023-10-12 22:32:33,913][44959] Updated weights for policy 1, policy_version 60450 (0.0009) [2023-10-12 22:32:34,275][44959] Updated weights for policy 1, policy_version 60460 (0.0007) [2023-10-12 22:32:34,640][44959] Updated weights for policy 1, policy_version 60470 (0.0007) [2023-10-12 22:32:35,009][44959] Updated weights for policy 1, policy_version 60480 (0.0008) [2023-10-12 22:32:36,261][44958] Updated weights for policy 0, policy_version 60170 (0.0009) [2023-10-12 22:32:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123535360. Throughput: 0: 1650.1, 1: 1647.0. Samples: 30890788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:36,443][43579] Avg episode reward: [(0, '276.870'), (1, '284.540')] [2023-10-12 22:32:36,636][44958] Updated weights for policy 0, policy_version 60180 (0.0008) [2023-10-12 22:32:37,000][44958] Updated weights for policy 0, policy_version 60190 (0.0009) [2023-10-12 22:32:39,169][44959] Updated weights for policy 1, policy_version 60490 (0.0010) [2023-10-12 22:32:39,541][44959] Updated weights for policy 1, policy_version 60500 (0.0009) [2023-10-12 22:32:39,924][44959] Updated weights for policy 1, policy_version 60510 (0.0007) [2023-10-12 22:32:41,261][44958] Updated weights for policy 0, policy_version 60200 (0.0008) [2023-10-12 22:32:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123600896. Throughput: 0: 1642.3, 1: 1642.8. Samples: 30909714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:41,444][43579] Avg episode reward: [(0, '283.630'), (1, '286.890')] [2023-10-12 22:32:41,633][44958] Updated weights for policy 0, policy_version 60210 (0.0009) [2023-10-12 22:32:42,016][44958] Updated weights for policy 0, policy_version 60220 (0.0008) [2023-10-12 22:32:44,092][44959] Updated weights for policy 1, policy_version 60520 (0.0007) [2023-10-12 22:32:44,468][44959] Updated weights for policy 1, policy_version 60530 (0.0007) [2023-10-12 22:32:44,826][44959] Updated weights for policy 1, policy_version 60540 (0.0009) [2023-10-12 22:32:45,986][44958] Updated weights for policy 0, policy_version 60230 (0.0008) [2023-10-12 22:32:46,367][44958] Updated weights for policy 0, policy_version 60240 (0.0007) [2023-10-12 22:32:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 123666432. Throughput: 0: 1647.7, 1: 1651.5. Samples: 30929906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:46,443][43579] Avg episode reward: [(0, '283.140'), (1, '283.050')] [2023-10-12 22:32:46,738][44958] Updated weights for policy 0, policy_version 60250 (0.0007) [2023-10-12 22:32:49,148][44959] Updated weights for policy 1, policy_version 60550 (0.0009) [2023-10-12 22:32:49,516][44959] Updated weights for policy 1, policy_version 60560 (0.0008) [2023-10-12 22:32:49,890][44959] Updated weights for policy 1, policy_version 60570 (0.0008) [2023-10-12 22:32:50,983][44958] Updated weights for policy 0, policy_version 60260 (0.0007) [2023-10-12 22:32:51,362][44958] Updated weights for policy 0, policy_version 60270 (0.0007) [2023-10-12 22:32:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123731968. Throughput: 0: 1641.6, 1: 1642.9. Samples: 30939900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:51,443][43579] Avg episode reward: [(0, '284.870'), (1, '284.740')] [2023-10-12 22:32:51,735][44958] Updated weights for policy 0, policy_version 60280 (0.0008) [2023-10-12 22:32:54,119][44959] Updated weights for policy 1, policy_version 60580 (0.0008) [2023-10-12 22:32:54,492][44959] Updated weights for policy 1, policy_version 60590 (0.0007) [2023-10-12 22:32:54,852][44959] Updated weights for policy 1, policy_version 60600 (0.0007) [2023-10-12 22:32:55,911][44958] Updated weights for policy 0, policy_version 60290 (0.0009) [2023-10-12 22:32:56,295][44958] Updated weights for policy 0, policy_version 60300 (0.0008) [2023-10-12 22:32:56,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123797504. Throughput: 0: 1638.5, 1: 1645.9. Samples: 30959270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:32:56,444][43579] Avg episode reward: [(0, '285.060'), (1, '281.900')] [2023-10-12 22:32:56,672][44958] Updated weights for policy 0, policy_version 60310 (0.0007) [2023-10-12 22:32:57,043][44958] Updated weights for policy 0, policy_version 60320 (0.0010) [2023-10-12 22:32:58,967][44959] Updated weights for policy 1, policy_version 60610 (0.0007) [2023-10-12 22:32:59,327][44959] Updated weights for policy 1, policy_version 60620 (0.0009) [2023-10-12 22:32:59,695][44959] Updated weights for policy 1, policy_version 60630 (0.0008) [2023-10-12 22:33:00,064][44959] Updated weights for policy 1, policy_version 60640 (0.0007) [2023-10-12 22:33:01,145][44958] Updated weights for policy 0, policy_version 60330 (0.0009) [2023-10-12 22:33:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 123863040. Throughput: 0: 1638.5, 1: 1655.5. Samples: 30979368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:01,443][43579] Avg episode reward: [(0, '281.710'), (1, '277.160')] [2023-10-12 22:33:01,521][44958] Updated weights for policy 0, policy_version 60340 (0.0007) [2023-10-12 22:33:01,891][44958] Updated weights for policy 0, policy_version 60350 (0.0009) [2023-10-12 22:33:04,384][44959] Updated weights for policy 1, policy_version 60650 (0.0010) [2023-10-12 22:33:04,762][44959] Updated weights for policy 1, policy_version 60660 (0.0010) [2023-10-12 22:33:05,126][44959] Updated weights for policy 1, policy_version 60670 (0.0009) [2023-10-12 22:33:05,987][44958] Updated weights for policy 0, policy_version 60360 (0.0008) [2023-10-12 22:33:06,359][44958] Updated weights for policy 0, policy_version 60370 (0.0010) [2023-10-12 22:33:06,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123928576. Throughput: 0: 1639.7, 1: 1654.3. Samples: 30989800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:06,444][43579] Avg episode reward: [(0, '283.190'), (1, '277.360')] [2023-10-12 22:33:06,737][44958] Updated weights for policy 0, policy_version 60380 (0.0009) [2023-10-12 22:33:09,219][44959] Updated weights for policy 1, policy_version 60680 (0.0011) [2023-10-12 22:33:09,592][44959] Updated weights for policy 1, policy_version 60690 (0.0007) [2023-10-12 22:33:09,953][44959] Updated weights for policy 1, policy_version 60700 (0.0009) [2023-10-12 22:33:11,027][44958] Updated weights for policy 0, policy_version 60390 (0.0008) [2023-10-12 22:33:11,398][44958] Updated weights for policy 0, policy_version 60400 (0.0009) [2023-10-12 22:33:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 123994112. Throughput: 0: 1643.5, 1: 1651.3. Samples: 31009000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:11,443][43579] Avg episode reward: [(0, '283.740'), (1, '275.450')] [2023-10-12 22:33:11,774][44958] Updated weights for policy 0, policy_version 60410 (0.0007) [2023-10-12 22:33:14,118][44959] Updated weights for policy 1, policy_version 60710 (0.0008) [2023-10-12 22:33:14,480][44959] Updated weights for policy 1, policy_version 60720 (0.0007) [2023-10-12 22:33:14,845][44959] Updated weights for policy 1, policy_version 60730 (0.0007) [2023-10-12 22:33:15,925][44958] Updated weights for policy 0, policy_version 60420 (0.0009) [2023-10-12 22:33:16,296][44958] Updated weights for policy 0, policy_version 60430 (0.0010) [2023-10-12 22:33:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 124059648. Throughput: 0: 1639.8, 1: 1651.1. Samples: 31028794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:16,443][43579] Avg episode reward: [(0, '280.890'), (1, '274.650')] [2023-10-12 22:33:16,674][44958] Updated weights for policy 0, policy_version 60440 (0.0010) [2023-10-12 22:33:19,225][44959] Updated weights for policy 1, policy_version 60740 (0.0009) [2023-10-12 22:33:19,587][44959] Updated weights for policy 1, policy_version 60750 (0.0007) [2023-10-12 22:33:19,960][44959] Updated weights for policy 1, policy_version 60760 (0.0007) [2023-10-12 22:33:21,030][44958] Updated weights for policy 0, policy_version 60450 (0.0009) [2023-10-12 22:33:21,439][44958] Updated weights for policy 0, policy_version 60460 (0.0010) [2023-10-12 22:33:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 124125184. Throughput: 0: 1638.5, 1: 1653.4. Samples: 31038924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:21,443][43579] Avg episode reward: [(0, '280.880'), (1, '277.100')] [2023-10-12 22:33:21,810][44958] Updated weights for policy 0, policy_version 60470 (0.0007) [2023-10-12 22:33:22,182][44958] Updated weights for policy 0, policy_version 60480 (0.0009) [2023-10-12 22:33:23,911][44959] Updated weights for policy 1, policy_version 60770 (0.0007) [2023-10-12 22:33:24,284][44959] Updated weights for policy 1, policy_version 60780 (0.0008) [2023-10-12 22:33:24,648][44959] Updated weights for policy 1, policy_version 60790 (0.0007) [2023-10-12 22:33:25,003][44959] Updated weights for policy 1, policy_version 60800 (0.0008) [2023-10-12 22:33:26,164][44958] Updated weights for policy 0, policy_version 60490 (0.0011) [2023-10-12 22:33:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 124190720. Throughput: 0: 1643.1, 1: 1650.7. Samples: 31057934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:26,444][43579] Avg episode reward: [(0, '280.910'), (1, '269.300')] [2023-10-12 22:33:26,533][44958] Updated weights for policy 0, policy_version 60500 (0.0010) [2023-10-12 22:33:26,909][44958] Updated weights for policy 0, policy_version 60510 (0.0008) [2023-10-12 22:33:29,190][44959] Updated weights for policy 1, policy_version 60810 (0.0010) [2023-10-12 22:33:29,562][44959] Updated weights for policy 1, policy_version 60820 (0.0010) [2023-10-12 22:33:29,932][44959] Updated weights for policy 1, policy_version 60830 (0.0008) [2023-10-12 22:33:31,085][44958] Updated weights for policy 0, policy_version 60520 (0.0008) [2023-10-12 22:33:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 124256256. Throughput: 0: 1636.0, 1: 1650.6. Samples: 31077802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:31,443][43579] Avg episode reward: [(0, '280.310'), (1, '268.610')] [2023-10-12 22:33:31,466][44958] Updated weights for policy 0, policy_version 60530 (0.0008) [2023-10-12 22:33:31,834][44958] Updated weights for policy 0, policy_version 60540 (0.0009) [2023-10-12 22:33:33,984][44959] Updated weights for policy 1, policy_version 60840 (0.0008) [2023-10-12 22:33:34,357][44959] Updated weights for policy 1, policy_version 60850 (0.0007) [2023-10-12 22:33:34,729][44959] Updated weights for policy 1, policy_version 60860 (0.0007) [2023-10-12 22:33:35,954][44958] Updated weights for policy 0, policy_version 60550 (0.0008) [2023-10-12 22:33:36,323][44958] Updated weights for policy 0, policy_version 60560 (0.0008) [2023-10-12 22:33:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 124321792. Throughput: 0: 1638.6, 1: 1650.7. Samples: 31087916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:36,444][43579] Avg episode reward: [(0, '277.450'), (1, '264.320')] [2023-10-12 22:33:36,695][44958] Updated weights for policy 0, policy_version 60570 (0.0008) [2023-10-12 22:33:38,929][44959] Updated weights for policy 1, policy_version 60870 (0.0008) [2023-10-12 22:33:39,295][44959] Updated weights for policy 1, policy_version 60880 (0.0009) [2023-10-12 22:33:39,662][44959] Updated weights for policy 1, policy_version 60890 (0.0010) [2023-10-12 22:33:40,764][44958] Updated weights for policy 0, policy_version 60580 (0.0009) [2023-10-12 22:33:41,144][44958] Updated weights for policy 0, policy_version 60590 (0.0008) [2023-10-12 22:33:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 124387328. Throughput: 0: 1648.7, 1: 1647.6. Samples: 31107604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:41,444][43579] Avg episode reward: [(0, '273.360'), (1, '264.010')] [2023-10-12 22:33:41,506][44958] Updated weights for policy 0, policy_version 60600 (0.0008) [2023-10-12 22:33:43,788][44959] Updated weights for policy 1, policy_version 60900 (0.0008) [2023-10-12 22:33:44,159][44959] Updated weights for policy 1, policy_version 60910 (0.0008) [2023-10-12 22:33:44,526][44959] Updated weights for policy 1, policy_version 60920 (0.0008) [2023-10-12 22:33:45,613][44958] Updated weights for policy 0, policy_version 60610 (0.0008) [2023-10-12 22:33:45,981][44958] Updated weights for policy 0, policy_version 60620 (0.0008) [2023-10-12 22:33:46,354][44958] Updated weights for policy 0, policy_version 60630 (0.0010) [2023-10-12 22:33:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 124452864. Throughput: 0: 1640.0, 1: 1651.1. Samples: 31127468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:46,444][43579] Avg episode reward: [(0, '274.880'), (1, '262.630')] [2023-10-12 22:33:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000060928_62390272.pth... [2023-10-12 22:33:46,488][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000059392_60817408.pth [2023-10-12 22:33:46,727][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000060640_62095360.pth... [2023-10-12 22:33:46,727][44958] Updated weights for policy 0, policy_version 60640 (0.0010) [2023-10-12 22:33:46,766][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000059104_60522496.pth [2023-10-12 22:33:48,572][44959] Updated weights for policy 1, policy_version 60930 (0.0008) [2023-10-12 22:33:48,941][44959] Updated weights for policy 1, policy_version 60940 (0.0010) [2023-10-12 22:33:49,318][44959] Updated weights for policy 1, policy_version 60950 (0.0008) [2023-10-12 22:33:49,680][44959] Updated weights for policy 1, policy_version 60960 (0.0011) [2023-10-12 22:33:51,122][44958] Updated weights for policy 0, policy_version 60650 (0.0007) [2023-10-12 22:33:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 124518400. Throughput: 0: 1640.9, 1: 1640.4. Samples: 31137460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:51,443][43579] Avg episode reward: [(0, '275.100'), (1, '267.240')] [2023-10-12 22:33:51,495][44958] Updated weights for policy 0, policy_version 60660 (0.0007) [2023-10-12 22:33:51,867][44958] Updated weights for policy 0, policy_version 60670 (0.0008) [2023-10-12 22:33:53,961][44959] Updated weights for policy 1, policy_version 60970 (0.0007) [2023-10-12 22:33:54,324][44959] Updated weights for policy 1, policy_version 60980 (0.0007) [2023-10-12 22:33:54,691][44959] Updated weights for policy 1, policy_version 60990 (0.0007) [2023-10-12 22:33:55,990][44958] Updated weights for policy 0, policy_version 60680 (0.0007) [2023-10-12 22:33:56,361][44958] Updated weights for policy 0, policy_version 60690 (0.0007) [2023-10-12 22:33:56,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 124583936. Throughput: 0: 1639.7, 1: 1646.4. Samples: 31156872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:33:56,443][43579] Avg episode reward: [(0, '276.660'), (1, '272.600')] [2023-10-12 22:33:56,729][44958] Updated weights for policy 0, policy_version 60700 (0.0007) [2023-10-12 22:33:58,958][44959] Updated weights for policy 1, policy_version 61000 (0.0009) [2023-10-12 22:33:59,328][44959] Updated weights for policy 1, policy_version 61010 (0.0009) [2023-10-12 22:33:59,693][44959] Updated weights for policy 1, policy_version 61020 (0.0009) [2023-10-12 22:34:00,836][44958] Updated weights for policy 0, policy_version 60710 (0.0007) [2023-10-12 22:34:01,199][44958] Updated weights for policy 0, policy_version 60720 (0.0007) [2023-10-12 22:34:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 124649472. Throughput: 0: 1636.4, 1: 1650.4. Samples: 31176702. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) [2023-10-12 22:34:01,443][43579] Avg episode reward: [(0, '278.300'), (1, '268.440')] [2023-10-12 22:34:01,575][44958] Updated weights for policy 0, policy_version 60730 (0.0008) [2023-10-12 22:34:03,727][44959] Updated weights for policy 1, policy_version 61030 (0.0009) [2023-10-12 22:34:04,096][44959] Updated weights for policy 1, policy_version 61040 (0.0010) [2023-10-12 22:34:04,468][44959] Updated weights for policy 1, policy_version 61050 (0.0009) [2023-10-12 22:34:05,941][44958] Updated weights for policy 0, policy_version 60740 (0.0008) [2023-10-12 22:34:06,328][44958] Updated weights for policy 0, policy_version 60750 (0.0007) [2023-10-12 22:34:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 124715008. Throughput: 0: 1644.6, 1: 1645.4. Samples: 31186972. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) [2023-10-12 22:34:06,443][43579] Avg episode reward: [(0, '275.620'), (1, '274.790')] [2023-10-12 22:34:06,690][44958] Updated weights for policy 0, policy_version 60760 (0.0007) [2023-10-12 22:34:08,539][44959] Updated weights for policy 1, policy_version 61060 (0.0007) [2023-10-12 22:34:08,900][44959] Updated weights for policy 1, policy_version 61070 (0.0008) [2023-10-12 22:34:09,273][44959] Updated weights for policy 1, policy_version 61080 (0.0008) [2023-10-12 22:34:10,781][44958] Updated weights for policy 0, policy_version 60770 (0.0007) [2023-10-12 22:34:11,154][44958] Updated weights for policy 0, policy_version 60780 (0.0008) [2023-10-12 22:34:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 124780544. Throughput: 0: 1646.8, 1: 1655.2. Samples: 31206522. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) [2023-10-12 22:34:11,443][43579] Avg episode reward: [(0, '277.940'), (1, '278.180')] [2023-10-12 22:34:11,522][44958] Updated weights for policy 0, policy_version 60790 (0.0007) [2023-10-12 22:34:11,895][44958] Updated weights for policy 0, policy_version 60800 (0.0007) [2023-10-12 22:34:13,289][44959] Updated weights for policy 1, policy_version 61090 (0.0009) [2023-10-12 22:34:13,651][44959] Updated weights for policy 1, policy_version 61100 (0.0009) [2023-10-12 22:34:14,015][44959] Updated weights for policy 1, policy_version 61110 (0.0008) [2023-10-12 22:34:14,381][44959] Updated weights for policy 1, policy_version 61120 (0.0010) [2023-10-12 22:34:15,909][44958] Updated weights for policy 0, policy_version 60810 (0.0008) [2023-10-12 22:34:16,278][44958] Updated weights for policy 0, policy_version 60820 (0.0008) [2023-10-12 22:34:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 124846080. Throughput: 0: 1645.6, 1: 1658.5. Samples: 31226488. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) [2023-10-12 22:34:16,443][43579] Avg episode reward: [(0, '279.040'), (1, '270.130')] [2023-10-12 22:34:16,657][44958] Updated weights for policy 0, policy_version 60830 (0.0009) [2023-10-12 22:34:18,589][44959] Updated weights for policy 1, policy_version 61130 (0.0008) [2023-10-12 22:34:18,965][44959] Updated weights for policy 1, policy_version 61140 (0.0009) [2023-10-12 22:34:19,340][44959] Updated weights for policy 1, policy_version 61150 (0.0011) [2023-10-12 22:34:20,683][44958] Updated weights for policy 0, policy_version 60840 (0.0007) [2023-10-12 22:34:21,056][44958] Updated weights for policy 0, policy_version 60850 (0.0007) [2023-10-12 22:34:21,442][44958] Updated weights for policy 0, policy_version 60860 (0.0007) [2023-10-12 22:34:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 124911616. Throughput: 0: 1652.9, 1: 1646.9. Samples: 31236406. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) [2023-10-12 22:34:21,443][43579] Avg episode reward: [(0, '276.680'), (1, '271.070')] [2023-10-12 22:34:23,334][44959] Updated weights for policy 1, policy_version 61160 (0.0011) [2023-10-12 22:34:23,700][44959] Updated weights for policy 1, policy_version 61170 (0.0010) [2023-10-12 22:34:24,074][44959] Updated weights for policy 1, policy_version 61180 (0.0007) [2023-10-12 22:34:25,628][44958] Updated weights for policy 0, policy_version 60870 (0.0008) [2023-10-12 22:34:25,996][44958] Updated weights for policy 0, policy_version 60880 (0.0010) [2023-10-12 22:34:26,372][44958] Updated weights for policy 0, policy_version 60890 (0.0011) [2023-10-12 22:34:26,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 124977152. Throughput: 0: 1645.3, 1: 1658.0. Samples: 31256254. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) [2023-10-12 22:34:26,443][43579] Avg episode reward: [(0, '276.400'), (1, '271.670')] [2023-10-12 22:34:28,016][44959] Updated weights for policy 1, policy_version 61190 (0.0008) [2023-10-12 22:34:28,386][44959] Updated weights for policy 1, policy_version 61200 (0.0009) [2023-10-12 22:34:28,748][44959] Updated weights for policy 1, policy_version 61210 (0.0009) [2023-10-12 22:34:30,634][44958] Updated weights for policy 0, policy_version 60900 (0.0008) [2023-10-12 22:34:31,009][44958] Updated weights for policy 0, policy_version 60910 (0.0007) [2023-10-12 22:34:31,384][44958] Updated weights for policy 0, policy_version 60920 (0.0008) [2023-10-12 22:34:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 125042688. Throughput: 0: 1641.4, 1: 1656.0. Samples: 31275852. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) [2023-10-12 22:34:31,443][43579] Avg episode reward: [(0, '276.050'), (1, '274.310')] [2023-10-12 22:34:33,344][44959] Updated weights for policy 1, policy_version 61220 (0.0009) [2023-10-12 22:34:33,705][44959] Updated weights for policy 1, policy_version 61230 (0.0009) [2023-10-12 22:34:34,066][44959] Updated weights for policy 1, policy_version 61240 (0.0008) [2023-10-12 22:34:35,693][44958] Updated weights for policy 0, policy_version 60930 (0.0010) [2023-10-12 22:34:36,059][44958] Updated weights for policy 0, policy_version 60940 (0.0011) [2023-10-12 22:34:36,435][44958] Updated weights for policy 0, policy_version 60950 (0.0011) [2023-10-12 22:34:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 125108224. Throughput: 0: 1649.2, 1: 1646.4. Samples: 31285760. Policy #0 lag: (min: 3.0, avg: 3.0, max: 4.0) [2023-10-12 22:34:36,443][43579] Avg episode reward: [(0, '275.230'), (1, '270.510')] [2023-10-12 22:34:36,800][44958] Updated weights for policy 0, policy_version 60960 (0.0010) [2023-10-12 22:34:38,130][44959] Updated weights for policy 1, policy_version 61250 (0.0009) [2023-10-12 22:34:38,495][44959] Updated weights for policy 1, policy_version 61260 (0.0008) [2023-10-12 22:34:38,869][44959] Updated weights for policy 1, policy_version 61270 (0.0008) [2023-10-12 22:34:39,229][44959] Updated weights for policy 1, policy_version 61280 (0.0009) [2023-10-12 22:34:40,917][44958] Updated weights for policy 0, policy_version 60970 (0.0008) [2023-10-12 22:34:41,299][44958] Updated weights for policy 0, policy_version 60980 (0.0010) [2023-10-12 22:34:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 125173760. Throughput: 0: 1642.6, 1: 1660.7. Samples: 31305518. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-12 22:34:41,443][43579] Avg episode reward: [(0, '280.940'), (1, '274.750')] [2023-10-12 22:34:41,668][44958] Updated weights for policy 0, policy_version 60990 (0.0007) [2023-10-12 22:34:43,361][44959] Updated weights for policy 1, policy_version 61290 (0.0009) [2023-10-12 22:34:43,732][44959] Updated weights for policy 1, policy_version 61300 (0.0011) [2023-10-12 22:34:44,103][44959] Updated weights for policy 1, policy_version 61310 (0.0008) [2023-10-12 22:34:45,800][44958] Updated weights for policy 0, policy_version 61000 (0.0008) [2023-10-12 22:34:46,174][44958] Updated weights for policy 0, policy_version 61010 (0.0009) [2023-10-12 22:34:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 125239296. Throughput: 0: 1644.4, 1: 1654.9. Samples: 31325168. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-12 22:34:46,443][43579] Avg episode reward: [(0, '280.500'), (1, '279.700')] [2023-10-12 22:34:46,548][44958] Updated weights for policy 0, policy_version 61020 (0.0011) [2023-10-12 22:34:48,332][44959] Updated weights for policy 1, policy_version 61320 (0.0009) [2023-10-12 22:34:48,698][44959] Updated weights for policy 1, policy_version 61330 (0.0010) [2023-10-12 22:34:49,061][44959] Updated weights for policy 1, policy_version 61340 (0.0009) [2023-10-12 22:34:50,697][44958] Updated weights for policy 0, policy_version 61030 (0.0009) [2023-10-12 22:34:51,092][44958] Updated weights for policy 0, policy_version 61040 (0.0008) [2023-10-12 22:34:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 125304832. Throughput: 0: 1650.1, 1: 1638.5. Samples: 31334960. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-12 22:34:51,443][43579] Avg episode reward: [(0, '273.160'), (1, '278.850')] [2023-10-12 22:34:51,471][44958] Updated weights for policy 0, policy_version 61050 (0.0008) [2023-10-12 22:34:53,167][44959] Updated weights for policy 1, policy_version 61350 (0.0008) [2023-10-12 22:34:53,532][44959] Updated weights for policy 1, policy_version 61360 (0.0010) [2023-10-12 22:34:53,903][44959] Updated weights for policy 1, policy_version 61370 (0.0009) [2023-10-12 22:34:55,687][44958] Updated weights for policy 0, policy_version 61060 (0.0009) [2023-10-12 22:34:56,054][44958] Updated weights for policy 0, policy_version 61070 (0.0009) [2023-10-12 22:34:56,430][44958] Updated weights for policy 0, policy_version 61080 (0.0009) [2023-10-12 22:34:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 125370368. Throughput: 0: 1648.3, 1: 1650.1. Samples: 31354950. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-12 22:34:56,443][43579] Avg episode reward: [(0, '276.080'), (1, '282.210')] [2023-10-12 22:34:57,994][44959] Updated weights for policy 1, policy_version 61380 (0.0010) [2023-10-12 22:34:58,358][44959] Updated weights for policy 1, policy_version 61390 (0.0008) [2023-10-12 22:34:58,727][44959] Updated weights for policy 1, policy_version 61400 (0.0007) [2023-10-12 22:35:00,483][44958] Updated weights for policy 0, policy_version 61090 (0.0008) [2023-10-12 22:35:00,847][44958] Updated weights for policy 0, policy_version 61100 (0.0009) [2023-10-12 22:35:01,229][44958] Updated weights for policy 0, policy_version 61110 (0.0008) [2023-10-12 22:35:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 125435904. Throughput: 0: 1639.8, 1: 1643.8. Samples: 31374252. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-12 22:35:01,443][43579] Avg episode reward: [(0, '271.680'), (1, '283.340')] [2023-10-12 22:35:01,605][44958] Updated weights for policy 0, policy_version 61120 (0.0008) [2023-10-12 22:35:03,203][44959] Updated weights for policy 1, policy_version 61410 (0.0008) [2023-10-12 22:35:03,566][44959] Updated weights for policy 1, policy_version 61420 (0.0010) [2023-10-12 22:35:03,937][44959] Updated weights for policy 1, policy_version 61430 (0.0008) [2023-10-12 22:35:04,310][44959] Updated weights for policy 1, policy_version 61440 (0.0010) [2023-10-12 22:35:05,677][44958] Updated weights for policy 0, policy_version 61130 (0.0011) [2023-10-12 22:35:06,046][44958] Updated weights for policy 0, policy_version 61140 (0.0010) [2023-10-12 22:35:06,419][44958] Updated weights for policy 0, policy_version 61150 (0.0008) [2023-10-12 22:35:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 125501440. Throughput: 0: 1642.0, 1: 1639.3. Samples: 31384064. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-12 22:35:06,443][43579] Avg episode reward: [(0, '275.650'), (1, '282.020')] [2023-10-12 22:35:08,422][44959] Updated weights for policy 1, policy_version 61450 (0.0010) [2023-10-12 22:35:08,781][44959] Updated weights for policy 1, policy_version 61460 (0.0010) [2023-10-12 22:35:09,145][44959] Updated weights for policy 1, policy_version 61470 (0.0007) [2023-10-12 22:35:10,474][44958] Updated weights for policy 0, policy_version 61160 (0.0008) [2023-10-12 22:35:10,845][44958] Updated weights for policy 0, policy_version 61170 (0.0008) [2023-10-12 22:35:11,221][44958] Updated weights for policy 0, policy_version 61180 (0.0009) [2023-10-12 22:35:11,442][43579] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 125599744. Throughput: 0: 1646.8, 1: 1639.6. Samples: 31404138. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-12 22:35:11,443][43579] Avg episode reward: [(0, '272.270'), (1, '282.720')] [2023-10-12 22:35:13,368][44959] Updated weights for policy 1, policy_version 61480 (0.0007) [2023-10-12 22:35:13,738][44959] Updated weights for policy 1, policy_version 61490 (0.0008) [2023-10-12 22:35:14,106][44959] Updated weights for policy 1, policy_version 61500 (0.0008) [2023-10-12 22:35:15,371][44958] Updated weights for policy 0, policy_version 61190 (0.0009) [2023-10-12 22:35:15,749][44958] Updated weights for policy 0, policy_version 61200 (0.0009) [2023-10-12 22:35:16,126][44958] Updated weights for policy 0, policy_version 61210 (0.0008) [2023-10-12 22:35:16,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 125665280. Throughput: 0: 1641.6, 1: 1640.9. Samples: 31423568. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-12 22:35:16,443][43579] Avg episode reward: [(0, '274.360'), (1, '281.790')] [2023-10-12 22:35:18,300][44959] Updated weights for policy 1, policy_version 61510 (0.0008) [2023-10-12 22:35:18,669][44959] Updated weights for policy 1, policy_version 61520 (0.0009) [2023-10-12 22:35:19,040][44959] Updated weights for policy 1, policy_version 61530 (0.0009) [2023-10-12 22:35:20,319][44958] Updated weights for policy 0, policy_version 61220 (0.0007) [2023-10-12 22:35:20,686][44958] Updated weights for policy 0, policy_version 61230 (0.0008) [2023-10-12 22:35:21,057][44958] Updated weights for policy 0, policy_version 61240 (0.0009) [2023-10-12 22:35:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 125730816. Throughput: 0: 1652.1, 1: 1637.5. Samples: 31433794. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-12 22:35:21,443][43579] Avg episode reward: [(0, '279.610'), (1, '285.970')] [2023-10-12 22:35:22,987][44959] Updated weights for policy 1, policy_version 61540 (0.0007) [2023-10-12 22:35:23,350][44959] Updated weights for policy 1, policy_version 61550 (0.0008) [2023-10-12 22:35:23,722][44959] Updated weights for policy 1, policy_version 61560 (0.0009) [2023-10-12 22:35:25,372][44958] Updated weights for policy 0, policy_version 61250 (0.0010) [2023-10-12 22:35:25,744][44958] Updated weights for policy 0, policy_version 61260 (0.0011) [2023-10-12 22:35:26,115][44958] Updated weights for policy 0, policy_version 61270 (0.0009) [2023-10-12 22:35:26,442][43579] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 125763584. Throughput: 0: 1656.5, 1: 1639.6. Samples: 31453846. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-12 22:35:26,443][43579] Avg episode reward: [(0, '275.190'), (1, '283.510')] [2023-10-12 22:35:26,488][44958] Updated weights for policy 0, policy_version 61280 (0.0010) [2023-10-12 22:35:28,178][44959] Updated weights for policy 1, policy_version 61570 (0.0010) [2023-10-12 22:35:28,540][44959] Updated weights for policy 1, policy_version 61580 (0.0009) [2023-10-12 22:35:28,913][44959] Updated weights for policy 1, policy_version 61590 (0.0008) [2023-10-12 22:35:29,281][44959] Updated weights for policy 1, policy_version 61600 (0.0009) [2023-10-12 22:35:30,527][44958] Updated weights for policy 0, policy_version 61290 (0.0008) [2023-10-12 22:35:30,893][44958] Updated weights for policy 0, policy_version 61300 (0.0009) [2023-10-12 22:35:31,271][44958] Updated weights for policy 0, policy_version 61310 (0.0007) [2023-10-12 22:35:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 125861888. Throughput: 0: 1645.2, 1: 1641.9. Samples: 31473086. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-12 22:35:31,444][43579] Avg episode reward: [(0, '273.000'), (1, '280.150')] [2023-10-12 22:35:33,259][44959] Updated weights for policy 1, policy_version 61610 (0.0009) [2023-10-12 22:35:33,632][44959] Updated weights for policy 1, policy_version 61620 (0.0008) [2023-10-12 22:35:34,012][44959] Updated weights for policy 1, policy_version 61630 (0.0009) [2023-10-12 22:35:35,508][44958] Updated weights for policy 0, policy_version 61320 (0.0007) [2023-10-12 22:35:35,885][44958] Updated weights for policy 0, policy_version 61330 (0.0008) [2023-10-12 22:35:36,260][44958] Updated weights for policy 0, policy_version 61340 (0.0008) [2023-10-12 22:35:36,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 125927424. Throughput: 0: 1647.4, 1: 1641.0. Samples: 31482938. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-12 22:35:36,443][43579] Avg episode reward: [(0, '272.760'), (1, '283.660')] [2023-10-12 22:35:38,185][44959] Updated weights for policy 1, policy_version 61640 (0.0010) [2023-10-12 22:35:38,554][44959] Updated weights for policy 1, policy_version 61650 (0.0007) [2023-10-12 22:35:38,927][44959] Updated weights for policy 1, policy_version 61660 (0.0008) [2023-10-12 22:35:40,496][44958] Updated weights for policy 0, policy_version 61350 (0.0008) [2023-10-12 22:35:40,866][44958] Updated weights for policy 0, policy_version 61360 (0.0009) [2023-10-12 22:35:41,236][44958] Updated weights for policy 0, policy_version 61370 (0.0010) [2023-10-12 22:35:41,442][43579] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 125960192. Throughput: 0: 1650.3, 1: 1641.2. Samples: 31503070. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-12 22:35:41,443][43579] Avg episode reward: [(0, '276.450'), (1, '284.930')] [2023-10-12 22:35:43,024][44959] Updated weights for policy 1, policy_version 61670 (0.0010) [2023-10-12 22:35:43,406][44959] Updated weights for policy 1, policy_version 61680 (0.0007) [2023-10-12 22:35:43,764][44959] Updated weights for policy 1, policy_version 61690 (0.0010) [2023-10-12 22:35:45,343][44958] Updated weights for policy 0, policy_version 61380 (0.0008) [2023-10-12 22:35:45,718][44958] Updated weights for policy 0, policy_version 61390 (0.0009) [2023-10-12 22:35:46,099][44958] Updated weights for policy 0, policy_version 61400 (0.0009) [2023-10-12 22:35:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13653.2, 300 sec: 13218.3). Total num frames: 126058496. Throughput: 0: 1646.6, 1: 1644.2. Samples: 31522336. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-12 22:35:46,444][43579] Avg episode reward: [(0, '265.460'), (1, '284.210')] [2023-10-12 22:35:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000061696_63176704.pth... [2023-10-12 22:35:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000061408_62881792.pth... [2023-10-12 22:35:46,490][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000060160_61603840.pth [2023-10-12 22:35:46,491][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000059872_61308928.pth [2023-10-12 22:35:46,494][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000061696_63176704.pth [2023-10-12 22:35:46,495][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000061408_62881792.pth [2023-10-12 22:35:48,152][44959] Updated weights for policy 1, policy_version 61700 (0.0010) [2023-10-12 22:35:48,509][44959] Updated weights for policy 1, policy_version 61710 (0.0007) [2023-10-12 22:35:48,882][44959] Updated weights for policy 1, policy_version 61720 (0.0007) [2023-10-12 22:35:50,451][44958] Updated weights for policy 0, policy_version 61410 (0.0007) [2023-10-12 22:35:50,828][44958] Updated weights for policy 0, policy_version 61420 (0.0007) [2023-10-12 22:35:51,202][44958] Updated weights for policy 0, policy_version 61430 (0.0008) [2023-10-12 22:35:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 126091264. Throughput: 0: 1646.7, 1: 1645.9. Samples: 31532230. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-12 22:35:51,443][43579] Avg episode reward: [(0, '267.450'), (1, '284.910')] [2023-10-12 22:35:51,574][44958] Updated weights for policy 0, policy_version 61440 (0.0007) [2023-10-12 22:35:52,949][44959] Updated weights for policy 1, policy_version 61730 (0.0008) [2023-10-12 22:35:53,317][44959] Updated weights for policy 1, policy_version 61740 (0.0011) [2023-10-12 22:35:53,681][44959] Updated weights for policy 1, policy_version 61750 (0.0010) [2023-10-12 22:35:54,050][44959] Updated weights for policy 1, policy_version 61760 (0.0009) [2023-10-12 22:35:55,630][44958] Updated weights for policy 0, policy_version 61450 (0.0010) [2023-10-12 22:35:56,009][44958] Updated weights for policy 0, policy_version 61460 (0.0008) [2023-10-12 22:35:56,379][44958] Updated weights for policy 0, policy_version 61470 (0.0011) [2023-10-12 22:35:56,442][43579] Fps is (10 sec: 9830.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 126156800. Throughput: 0: 1640.3, 1: 1647.6. Samples: 31552094. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-12 22:35:56,443][43579] Avg episode reward: [(0, '270.660'), (1, '284.450')] [2023-10-12 22:35:58,209][44959] Updated weights for policy 1, policy_version 61770 (0.0009) [2023-10-12 22:35:58,569][44959] Updated weights for policy 1, policy_version 61780 (0.0010) [2023-10-12 22:35:58,936][44959] Updated weights for policy 1, policy_version 61790 (0.0010) [2023-10-12 22:36:00,747][44958] Updated weights for policy 0, policy_version 61480 (0.0008) [2023-10-12 22:36:01,123][44958] Updated weights for policy 0, policy_version 61490 (0.0009) [2023-10-12 22:36:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 126222336. Throughput: 0: 1641.3, 1: 1645.3. Samples: 31571466. Policy #0 lag: (min: 31.0, avg: 44.1, max: 63.0) [2023-10-12 22:36:01,443][43579] Avg episode reward: [(0, '275.140'), (1, '288.170')] [2023-10-12 22:36:01,504][44958] Updated weights for policy 0, policy_version 61500 (0.0007) [2023-10-12 22:36:03,265][44959] Updated weights for policy 1, policy_version 61800 (0.0009) [2023-10-12 22:36:03,626][44959] Updated weights for policy 1, policy_version 61810 (0.0010) [2023-10-12 22:36:03,994][44959] Updated weights for policy 1, policy_version 61820 (0.0008) [2023-10-12 22:36:05,367][44958] Updated weights for policy 0, policy_version 61510 (0.0007) [2023-10-12 22:36:05,728][44958] Updated weights for policy 0, policy_version 61520 (0.0008) [2023-10-12 22:36:06,099][44958] Updated weights for policy 0, policy_version 61530 (0.0011) [2023-10-12 22:36:06,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 126320640. Throughput: 0: 1632.5, 1: 1646.5. Samples: 31581352. Policy #0 lag: (min: 31.0, avg: 44.1, max: 63.0) [2023-10-12 22:36:06,443][43579] Avg episode reward: [(0, '274.450'), (1, '282.950')] [2023-10-12 22:36:08,091][44959] Updated weights for policy 1, policy_version 61830 (0.0007) [2023-10-12 22:36:08,453][44959] Updated weights for policy 1, policy_version 61840 (0.0008) [2023-10-12 22:36:08,822][44959] Updated weights for policy 1, policy_version 61850 (0.0009) [2023-10-12 22:36:10,298][44958] Updated weights for policy 0, policy_version 61540 (0.0010) [2023-10-12 22:36:10,671][44958] Updated weights for policy 0, policy_version 61550 (0.0009) [2023-10-12 22:36:11,051][44958] Updated weights for policy 0, policy_version 61560 (0.0008) [2023-10-12 22:36:11,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 126386176. Throughput: 0: 1631.9, 1: 1643.9. Samples: 31601256. Policy #0 lag: (min: 31.0, avg: 44.1, max: 63.0) [2023-10-12 22:36:11,443][43579] Avg episode reward: [(0, '277.950'), (1, '285.460')] [2023-10-12 22:36:12,926][44959] Updated weights for policy 1, policy_version 61860 (0.0007) [2023-10-12 22:36:13,296][44959] Updated weights for policy 1, policy_version 61870 (0.0009) [2023-10-12 22:36:13,662][44959] Updated weights for policy 1, policy_version 61880 (0.0007) [2023-10-12 22:36:15,392][44958] Updated weights for policy 0, policy_version 61570 (0.0010) [2023-10-12 22:36:15,763][44958] Updated weights for policy 0, policy_version 61580 (0.0009) [2023-10-12 22:36:16,124][44958] Updated weights for policy 0, policy_version 61590 (0.0008) [2023-10-12 22:36:16,443][43579] Fps is (10 sec: 9830.2, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 126418944. Throughput: 0: 1633.0, 1: 1650.2. Samples: 31620832. Policy #0 lag: (min: 31.0, avg: 44.1, max: 63.0) [2023-10-12 22:36:16,444][43579] Avg episode reward: [(0, '282.670'), (1, '285.470')] [2023-10-12 22:36:16,493][44958] Updated weights for policy 0, policy_version 61600 (0.0010) [2023-10-12 22:36:17,840][44959] Updated weights for policy 1, policy_version 61890 (0.0009) [2023-10-12 22:36:18,211][44959] Updated weights for policy 1, policy_version 61900 (0.0007) [2023-10-12 22:36:18,573][44959] Updated weights for policy 1, policy_version 61910 (0.0010) [2023-10-12 22:36:18,948][44959] Updated weights for policy 1, policy_version 61920 (0.0008) [2023-10-12 22:36:20,634][44958] Updated weights for policy 0, policy_version 61610 (0.0011) [2023-10-12 22:36:21,003][44958] Updated weights for policy 0, policy_version 61620 (0.0008) [2023-10-12 22:36:21,379][44958] Updated weights for policy 0, policy_version 61630 (0.0007) [2023-10-12 22:36:21,443][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 126484480. Throughput: 0: 1631.1, 1: 1647.6. Samples: 31630476. Policy #0 lag: (min: 31.0, avg: 44.1, max: 63.0) [2023-10-12 22:36:21,443][43579] Avg episode reward: [(0, '280.420'), (1, '288.650')] [2023-10-12 22:36:22,980][44959] Updated weights for policy 1, policy_version 61930 (0.0010) [2023-10-12 22:36:23,355][44959] Updated weights for policy 1, policy_version 61940 (0.0009) [2023-10-12 22:36:23,715][44959] Updated weights for policy 1, policy_version 61950 (0.0011) [2023-10-12 22:36:25,832][44958] Updated weights for policy 0, policy_version 61640 (0.0009) [2023-10-12 22:36:26,214][44958] Updated weights for policy 0, policy_version 61650 (0.0008) [2023-10-12 22:36:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 126550016. Throughput: 0: 1629.3, 1: 1648.1. Samples: 31650554. Policy #0 lag: (min: 31.0, avg: 44.1, max: 63.0) [2023-10-12 22:36:26,443][43579] Avg episode reward: [(0, '280.590'), (1, '287.180')] [2023-10-12 22:36:26,575][44958] Updated weights for policy 0, policy_version 61660 (0.0007) [2023-10-12 22:36:28,070][44959] Updated weights for policy 1, policy_version 61960 (0.0007) [2023-10-12 22:36:28,455][44959] Updated weights for policy 1, policy_version 61970 (0.0007) [2023-10-12 22:36:28,829][44959] Updated weights for policy 1, policy_version 61980 (0.0010) [2023-10-12 22:36:30,604][44958] Updated weights for policy 0, policy_version 61670 (0.0009) [2023-10-12 22:36:30,971][44958] Updated weights for policy 0, policy_version 61680 (0.0010) [2023-10-12 22:36:31,343][44958] Updated weights for policy 0, policy_version 61690 (0.0009) [2023-10-12 22:36:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 126615552. Throughput: 0: 1630.5, 1: 1648.6. Samples: 31669898. Policy #0 lag: (min: 31.0, avg: 44.1, max: 63.0) [2023-10-12 22:36:31,443][43579] Avg episode reward: [(0, '280.700'), (1, '285.190')] [2023-10-12 22:36:32,924][44959] Updated weights for policy 1, policy_version 61990 (0.0010) [2023-10-12 22:36:33,291][44959] Updated weights for policy 1, policy_version 62000 (0.0010) [2023-10-12 22:36:33,655][44959] Updated weights for policy 1, policy_version 62010 (0.0009) [2023-10-12 22:36:35,757][44958] Updated weights for policy 0, policy_version 61700 (0.0011) [2023-10-12 22:36:36,125][44958] Updated weights for policy 0, policy_version 61710 (0.0009) [2023-10-12 22:36:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 126681088. Throughput: 0: 1629.2, 1: 1642.4. Samples: 31679456. Policy #0 lag: (min: 31.0, avg: 44.1, max: 63.0) [2023-10-12 22:36:36,443][43579] Avg episode reward: [(0, '279.410'), (1, '288.910')] [2023-10-12 22:36:36,500][44958] Updated weights for policy 0, policy_version 61720 (0.0007) [2023-10-12 22:36:37,681][44959] Updated weights for policy 1, policy_version 62020 (0.0007) [2023-10-12 22:36:38,058][44959] Updated weights for policy 1, policy_version 62030 (0.0007) [2023-10-12 22:36:38,426][44959] Updated weights for policy 1, policy_version 62040 (0.0010) [2023-10-12 22:36:40,620][44958] Updated weights for policy 0, policy_version 61730 (0.0007) [2023-10-12 22:36:40,993][44958] Updated weights for policy 0, policy_version 61740 (0.0008) [2023-10-12 22:36:41,365][44958] Updated weights for policy 0, policy_version 61750 (0.0007) [2023-10-12 22:36:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 126746624. Throughput: 0: 1631.3, 1: 1648.4. Samples: 31699682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:36:41,444][43579] Avg episode reward: [(0, '281.320'), (1, '288.200')] [2023-10-12 22:36:41,738][44958] Updated weights for policy 0, policy_version 61760 (0.0007) [2023-10-12 22:36:42,724][44959] Updated weights for policy 1, policy_version 62050 (0.0008) [2023-10-12 22:36:43,088][44959] Updated weights for policy 1, policy_version 62060 (0.0010) [2023-10-12 22:36:43,457][44959] Updated weights for policy 1, policy_version 62070 (0.0011) [2023-10-12 22:36:43,829][44959] Updated weights for policy 1, policy_version 62080 (0.0011) [2023-10-12 22:36:45,971][44958] Updated weights for policy 0, policy_version 61770 (0.0007) [2023-10-12 22:36:46,337][44958] Updated weights for policy 0, policy_version 61780 (0.0011) [2023-10-12 22:36:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 126812160. Throughput: 0: 1637.9, 1: 1645.1. Samples: 31719200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:36:46,443][43579] Avg episode reward: [(0, '283.000'), (1, '283.790')] [2023-10-12 22:36:46,712][44958] Updated weights for policy 0, policy_version 61790 (0.0010) [2023-10-12 22:36:47,875][44959] Updated weights for policy 1, policy_version 62090 (0.0007) [2023-10-12 22:36:48,244][44959] Updated weights for policy 1, policy_version 62100 (0.0010) [2023-10-12 22:36:48,614][44959] Updated weights for policy 1, policy_version 62110 (0.0009) [2023-10-12 22:36:50,955][44958] Updated weights for policy 0, policy_version 61800 (0.0008) [2023-10-12 22:36:51,331][44958] Updated weights for policy 0, policy_version 61810 (0.0007) [2023-10-12 22:36:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 126877696. Throughput: 0: 1628.5, 1: 1640.4. Samples: 31728452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:36:51,444][43579] Avg episode reward: [(0, '282.960'), (1, '283.540')] [2023-10-12 22:36:51,698][44958] Updated weights for policy 0, policy_version 61820 (0.0007) [2023-10-12 22:36:52,771][44959] Updated weights for policy 1, policy_version 62120 (0.0008) [2023-10-12 22:36:53,136][44959] Updated weights for policy 1, policy_version 62130 (0.0007) [2023-10-12 22:36:53,501][44959] Updated weights for policy 1, policy_version 62140 (0.0009) [2023-10-12 22:36:55,775][44958] Updated weights for policy 0, policy_version 61830 (0.0010) [2023-10-12 22:36:56,147][44958] Updated weights for policy 0, policy_version 61840 (0.0010) [2023-10-12 22:36:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 126943232. Throughput: 0: 1626.8, 1: 1645.9. Samples: 31748526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:36:56,443][43579] Avg episode reward: [(0, '283.350'), (1, '286.420')] [2023-10-12 22:36:56,513][44958] Updated weights for policy 0, policy_version 61850 (0.0010) [2023-10-12 22:36:57,585][44959] Updated weights for policy 1, policy_version 62150 (0.0008) [2023-10-12 22:36:57,952][44959] Updated weights for policy 1, policy_version 62160 (0.0007) [2023-10-12 22:36:58,316][44959] Updated weights for policy 1, policy_version 62170 (0.0009) [2023-10-12 22:37:00,934][44958] Updated weights for policy 0, policy_version 61860 (0.0009) [2023-10-12 22:37:01,301][44958] Updated weights for policy 0, policy_version 61870 (0.0008) [2023-10-12 22:37:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 127008768. Throughput: 0: 1632.8, 1: 1642.8. Samples: 31768236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:37:01,444][43579] Avg episode reward: [(0, '285.210'), (1, '287.190')] [2023-10-12 22:37:01,672][44958] Updated weights for policy 0, policy_version 61880 (0.0007) [2023-10-12 22:37:02,636][44959] Updated weights for policy 1, policy_version 62180 (0.0010) [2023-10-12 22:37:03,008][44959] Updated weights for policy 1, policy_version 62190 (0.0007) [2023-10-12 22:37:03,376][44959] Updated weights for policy 1, policy_version 62200 (0.0008) [2023-10-12 22:37:05,702][44958] Updated weights for policy 0, policy_version 61890 (0.0009) [2023-10-12 22:37:06,077][44958] Updated weights for policy 0, policy_version 61900 (0.0009) [2023-10-12 22:37:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 127074304. Throughput: 0: 1625.6, 1: 1643.1. Samples: 31777566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:37:06,443][44958] Updated weights for policy 0, policy_version 61910 (0.0007) [2023-10-12 22:37:06,443][43579] Avg episode reward: [(0, '286.320'), (1, '287.200')] [2023-10-12 22:37:06,818][44958] Updated weights for policy 0, policy_version 61920 (0.0008) [2023-10-12 22:37:07,496][44959] Updated weights for policy 1, policy_version 62210 (0.0008) [2023-10-12 22:37:07,866][44959] Updated weights for policy 1, policy_version 62220 (0.0007) [2023-10-12 22:37:08,241][44959] Updated weights for policy 1, policy_version 62230 (0.0008) [2023-10-12 22:37:08,615][44959] Updated weights for policy 1, policy_version 62240 (0.0008) [2023-10-12 22:37:11,004][44958] Updated weights for policy 0, policy_version 61930 (0.0007) [2023-10-12 22:37:11,378][44958] Updated weights for policy 0, policy_version 61940 (0.0007) [2023-10-12 22:37:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 127139840. Throughput: 0: 1627.0, 1: 1647.9. Samples: 31797926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:37:11,443][43579] Avg episode reward: [(0, '284.900'), (1, '285.320')] [2023-10-12 22:37:11,746][44958] Updated weights for policy 0, policy_version 61950 (0.0007) [2023-10-12 22:37:12,774][44959] Updated weights for policy 1, policy_version 62250 (0.0007) [2023-10-12 22:37:13,151][44959] Updated weights for policy 1, policy_version 62260 (0.0008) [2023-10-12 22:37:13,516][44959] Updated weights for policy 1, policy_version 62270 (0.0008) [2023-10-12 22:37:16,013][44958] Updated weights for policy 0, policy_version 61960 (0.0010) [2023-10-12 22:37:16,387][44958] Updated weights for policy 0, policy_version 61970 (0.0009) [2023-10-12 22:37:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127205376. Throughput: 0: 1630.4, 1: 1652.3. Samples: 31817618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:37:16,444][43579] Avg episode reward: [(0, '282.150'), (1, '290.040')] [2023-10-12 22:37:16,761][44958] Updated weights for policy 0, policy_version 61980 (0.0007) [2023-10-12 22:37:17,649][44959] Updated weights for policy 1, policy_version 62280 (0.0009) [2023-10-12 22:37:18,018][44959] Updated weights for policy 1, policy_version 62290 (0.0007) [2023-10-12 22:37:18,379][44959] Updated weights for policy 1, policy_version 62300 (0.0008) [2023-10-12 22:37:20,957][44958] Updated weights for policy 0, policy_version 61990 (0.0008) [2023-10-12 22:37:21,325][44958] Updated weights for policy 0, policy_version 62000 (0.0007) [2023-10-12 22:37:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127270912. Throughput: 0: 1625.3, 1: 1650.9. Samples: 31826888. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) [2023-10-12 22:37:21,444][43579] Avg episode reward: [(0, '283.400'), (1, '284.380')] [2023-10-12 22:37:21,706][44958] Updated weights for policy 0, policy_version 62010 (0.0008) [2023-10-12 22:37:22,656][44959] Updated weights for policy 1, policy_version 62310 (0.0008) [2023-10-12 22:37:23,015][44959] Updated weights for policy 1, policy_version 62320 (0.0008) [2023-10-12 22:37:23,395][44959] Updated weights for policy 1, policy_version 62330 (0.0009) [2023-10-12 22:37:25,855][44958] Updated weights for policy 0, policy_version 62020 (0.0009) [2023-10-12 22:37:26,230][44958] Updated weights for policy 0, policy_version 62030 (0.0009) [2023-10-12 22:37:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127336448. Throughput: 0: 1624.3, 1: 1649.8. Samples: 31847016. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) [2023-10-12 22:37:26,443][43579] Avg episode reward: [(0, '280.020'), (1, '284.020')] [2023-10-12 22:37:26,604][44958] Updated weights for policy 0, policy_version 62040 (0.0009) [2023-10-12 22:37:27,644][44959] Updated weights for policy 1, policy_version 62340 (0.0009) [2023-10-12 22:37:28,017][44959] Updated weights for policy 1, policy_version 62350 (0.0008) [2023-10-12 22:37:28,391][44959] Updated weights for policy 1, policy_version 62360 (0.0007) [2023-10-12 22:37:30,947][44958] Updated weights for policy 0, policy_version 62050 (0.0010) [2023-10-12 22:37:31,315][44958] Updated weights for policy 0, policy_version 62060 (0.0007) [2023-10-12 22:37:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127401984. Throughput: 0: 1625.1, 1: 1649.4. Samples: 31866554. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) [2023-10-12 22:37:31,444][43579] Avg episode reward: [(0, '277.160'), (1, '282.130')] [2023-10-12 22:37:31,694][44958] Updated weights for policy 0, policy_version 62070 (0.0009) [2023-10-12 22:37:32,071][44958] Updated weights for policy 0, policy_version 62080 (0.0008) [2023-10-12 22:37:32,458][44959] Updated weights for policy 1, policy_version 62370 (0.0008) [2023-10-12 22:37:32,832][44959] Updated weights for policy 1, policy_version 62380 (0.0009) [2023-10-12 22:37:33,203][44959] Updated weights for policy 1, policy_version 62390 (0.0009) [2023-10-12 22:37:33,559][44959] Updated weights for policy 1, policy_version 62400 (0.0009) [2023-10-12 22:37:36,282][44958] Updated weights for policy 0, policy_version 62090 (0.0008) [2023-10-12 22:37:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 127467520. Throughput: 0: 1625.4, 1: 1649.6. Samples: 31875828. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) [2023-10-12 22:37:36,444][43579] Avg episode reward: [(0, '278.500'), (1, '279.450')] [2023-10-12 22:37:36,652][44958] Updated weights for policy 0, policy_version 62100 (0.0007) [2023-10-12 22:37:37,021][44958] Updated weights for policy 0, policy_version 62110 (0.0009) [2023-10-12 22:37:37,716][44959] Updated weights for policy 1, policy_version 62410 (0.0008) [2023-10-12 22:37:38,073][44959] Updated weights for policy 1, policy_version 62420 (0.0009) [2023-10-12 22:37:38,437][44959] Updated weights for policy 1, policy_version 62430 (0.0009) [2023-10-12 22:37:41,076][44958] Updated weights for policy 0, policy_version 62120 (0.0009) [2023-10-12 22:37:41,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 127533056. Throughput: 0: 1624.9, 1: 1655.7. Samples: 31896156. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) [2023-10-12 22:37:41,443][43579] Avg episode reward: [(0, '278.360'), (1, '275.470')] [2023-10-12 22:37:41,449][44958] Updated weights for policy 0, policy_version 62130 (0.0010) [2023-10-12 22:37:41,824][44958] Updated weights for policy 0, policy_version 62140 (0.0008) [2023-10-12 22:37:42,434][44959] Updated weights for policy 1, policy_version 62440 (0.0007) [2023-10-12 22:37:42,807][44959] Updated weights for policy 1, policy_version 62450 (0.0009) [2023-10-12 22:37:43,181][44959] Updated weights for policy 1, policy_version 62460 (0.0008) [2023-10-12 22:37:45,879][44958] Updated weights for policy 0, policy_version 62150 (0.0008) [2023-10-12 22:37:46,261][44958] Updated weights for policy 0, policy_version 62160 (0.0008) [2023-10-12 22:37:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127598592. Throughput: 0: 1629.5, 1: 1657.6. Samples: 31916154. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) [2023-10-12 22:37:46,443][43579] Avg episode reward: [(0, '278.110'), (1, '273.190')] [2023-10-12 22:37:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000062464_63963136.pth... [2023-10-12 22:37:46,489][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000060928_62390272.pth [2023-10-12 22:37:46,629][44958] Updated weights for policy 0, policy_version 62170 (0.0007) [2023-10-12 22:37:46,844][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000062176_63668224.pth... [2023-10-12 22:37:46,881][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000060640_62095360.pth [2023-10-12 22:37:47,289][44959] Updated weights for policy 1, policy_version 62470 (0.0007) [2023-10-12 22:37:47,655][44959] Updated weights for policy 1, policy_version 62480 (0.0007) [2023-10-12 22:37:48,025][44959] Updated weights for policy 1, policy_version 62490 (0.0008) [2023-10-12 22:37:50,883][44958] Updated weights for policy 0, policy_version 62180 (0.0008) [2023-10-12 22:37:51,258][44958] Updated weights for policy 0, policy_version 62190 (0.0007) [2023-10-12 22:37:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127664128. Throughput: 0: 1628.7, 1: 1659.4. Samples: 31925532. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) [2023-10-12 22:37:51,443][43579] Avg episode reward: [(0, '280.310'), (1, '270.410')] [2023-10-12 22:37:51,626][44958] Updated weights for policy 0, policy_version 62200 (0.0008) [2023-10-12 22:37:52,125][44959] Updated weights for policy 1, policy_version 62500 (0.0010) [2023-10-12 22:37:52,503][44959] Updated weights for policy 1, policy_version 62510 (0.0008) [2023-10-12 22:37:52,879][44959] Updated weights for policy 1, policy_version 62520 (0.0008) [2023-10-12 22:37:55,905][44958] Updated weights for policy 0, policy_version 62210 (0.0009) [2023-10-12 22:37:56,298][44958] Updated weights for policy 0, policy_version 62220 (0.0009) [2023-10-12 22:37:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127729664. Throughput: 0: 1627.3, 1: 1656.6. Samples: 31945702. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) [2023-10-12 22:37:56,443][43579] Avg episode reward: [(0, '282.890'), (1, '270.860')] [2023-10-12 22:37:56,658][44958] Updated weights for policy 0, policy_version 62230 (0.0010) [2023-10-12 22:37:57,032][44958] Updated weights for policy 0, policy_version 62240 (0.0010) [2023-10-12 22:37:57,279][44959] Updated weights for policy 1, policy_version 62530 (0.0010) [2023-10-12 22:37:57,692][44959] Updated weights for policy 1, policy_version 62540 (0.0011) [2023-10-12 22:37:58,067][44959] Updated weights for policy 1, policy_version 62550 (0.0011) [2023-10-12 22:37:58,434][44959] Updated weights for policy 1, policy_version 62560 (0.0009) [2023-10-12 22:38:01,183][44958] Updated weights for policy 0, policy_version 62250 (0.0007) [2023-10-12 22:38:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 127795200. Throughput: 0: 1636.3, 1: 1645.5. Samples: 31965296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:01,443][43579] Avg episode reward: [(0, '277.770'), (1, '269.520')] [2023-10-12 22:38:01,543][44958] Updated weights for policy 0, policy_version 62260 (0.0010) [2023-10-12 22:38:01,914][44958] Updated weights for policy 0, policy_version 62270 (0.0008) [2023-10-12 22:38:02,600][44959] Updated weights for policy 1, policy_version 62570 (0.0007) [2023-10-12 22:38:02,976][44959] Updated weights for policy 1, policy_version 62580 (0.0007) [2023-10-12 22:38:03,350][44959] Updated weights for policy 1, policy_version 62590 (0.0009) [2023-10-12 22:38:06,178][44958] Updated weights for policy 0, policy_version 62280 (0.0008) [2023-10-12 22:38:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127860736. Throughput: 0: 1630.4, 1: 1648.3. Samples: 31974426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:06,443][43579] Avg episode reward: [(0, '276.550'), (1, '271.890')] [2023-10-12 22:38:06,562][44958] Updated weights for policy 0, policy_version 62290 (0.0007) [2023-10-12 22:38:06,926][44958] Updated weights for policy 0, policy_version 62300 (0.0009) [2023-10-12 22:38:07,533][44959] Updated weights for policy 1, policy_version 62600 (0.0008) [2023-10-12 22:38:07,903][44959] Updated weights for policy 1, policy_version 62610 (0.0009) [2023-10-12 22:38:08,263][44959] Updated weights for policy 1, policy_version 62620 (0.0008) [2023-10-12 22:38:10,882][44958] Updated weights for policy 0, policy_version 62310 (0.0008) [2023-10-12 22:38:11,252][44958] Updated weights for policy 0, policy_version 62320 (0.0007) [2023-10-12 22:38:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127926272. Throughput: 0: 1633.9, 1: 1651.6. Samples: 31994862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:11,443][43579] Avg episode reward: [(0, '282.340'), (1, '271.400')] [2023-10-12 22:38:11,621][44958] Updated weights for policy 0, policy_version 62330 (0.0008) [2023-10-12 22:38:12,237][44959] Updated weights for policy 1, policy_version 62630 (0.0009) [2023-10-12 22:38:12,613][44959] Updated weights for policy 1, policy_version 62640 (0.0007) [2023-10-12 22:38:12,987][44959] Updated weights for policy 1, policy_version 62650 (0.0010) [2023-10-12 22:38:15,861][44958] Updated weights for policy 0, policy_version 62340 (0.0008) [2023-10-12 22:38:16,233][44958] Updated weights for policy 0, policy_version 62350 (0.0008) [2023-10-12 22:38:16,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 127991808. Throughput: 0: 1641.2, 1: 1658.2. Samples: 32015024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:16,444][43579] Avg episode reward: [(0, '274.690'), (1, '273.800')] [2023-10-12 22:38:16,615][44958] Updated weights for policy 0, policy_version 62360 (0.0011) [2023-10-12 22:38:17,124][44959] Updated weights for policy 1, policy_version 62660 (0.0009) [2023-10-12 22:38:17,501][44959] Updated weights for policy 1, policy_version 62670 (0.0008) [2023-10-12 22:38:17,859][44959] Updated weights for policy 1, policy_version 62680 (0.0010) [2023-10-12 22:38:20,808][44958] Updated weights for policy 0, policy_version 62370 (0.0010) [2023-10-12 22:38:21,183][44958] Updated weights for policy 0, policy_version 62380 (0.0011) [2023-10-12 22:38:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128057344. Throughput: 0: 1640.8, 1: 1657.5. Samples: 32024250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:21,443][43579] Avg episode reward: [(0, '271.970'), (1, '277.430')] [2023-10-12 22:38:21,547][44958] Updated weights for policy 0, policy_version 62390 (0.0011) [2023-10-12 22:38:21,928][44958] Updated weights for policy 0, policy_version 62400 (0.0011) [2023-10-12 22:38:22,055][44959] Updated weights for policy 1, policy_version 62690 (0.0008) [2023-10-12 22:38:22,417][44959] Updated weights for policy 1, policy_version 62700 (0.0008) [2023-10-12 22:38:22,788][44959] Updated weights for policy 1, policy_version 62710 (0.0007) [2023-10-12 22:38:23,165][44959] Updated weights for policy 1, policy_version 62720 (0.0009) [2023-10-12 22:38:26,088][44958] Updated weights for policy 0, policy_version 62410 (0.0008) [2023-10-12 22:38:26,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128122880. Throughput: 0: 1642.4, 1: 1655.9. Samples: 32044580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:26,443][43579] Avg episode reward: [(0, '274.180'), (1, '275.820')] [2023-10-12 22:38:26,466][44958] Updated weights for policy 0, policy_version 62420 (0.0008) [2023-10-12 22:38:26,833][44958] Updated weights for policy 0, policy_version 62430 (0.0010) [2023-10-12 22:38:27,155][44959] Updated weights for policy 1, policy_version 62730 (0.0008) [2023-10-12 22:38:27,521][44959] Updated weights for policy 1, policy_version 62740 (0.0009) [2023-10-12 22:38:27,894][44959] Updated weights for policy 1, policy_version 62750 (0.0008) [2023-10-12 22:38:31,065][44958] Updated weights for policy 0, policy_version 62440 (0.0007) [2023-10-12 22:38:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128188416. Throughput: 0: 1645.7, 1: 1655.4. Samples: 32064704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:31,443][43579] Avg episode reward: [(0, '277.030'), (1, '280.020')] [2023-10-12 22:38:31,447][44958] Updated weights for policy 0, policy_version 62450 (0.0008) [2023-10-12 22:38:31,814][44958] Updated weights for policy 0, policy_version 62460 (0.0008) [2023-10-12 22:38:32,014][44959] Updated weights for policy 1, policy_version 62760 (0.0009) [2023-10-12 22:38:32,390][44959] Updated weights for policy 1, policy_version 62770 (0.0008) [2023-10-12 22:38:32,759][44959] Updated weights for policy 1, policy_version 62780 (0.0007) [2023-10-12 22:38:35,965][44958] Updated weights for policy 0, policy_version 62470 (0.0007) [2023-10-12 22:38:36,328][44958] Updated weights for policy 0, policy_version 62480 (0.0008) [2023-10-12 22:38:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128253952. Throughput: 0: 1645.0, 1: 1656.1. Samples: 32074082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:36,443][43579] Avg episode reward: [(0, '275.560'), (1, '275.070')] [2023-10-12 22:38:36,703][44958] Updated weights for policy 0, policy_version 62490 (0.0008) [2023-10-12 22:38:36,751][44959] Updated weights for policy 1, policy_version 62790 (0.0008) [2023-10-12 22:38:37,118][44959] Updated weights for policy 1, policy_version 62800 (0.0009) [2023-10-12 22:38:37,493][44959] Updated weights for policy 1, policy_version 62810 (0.0009) [2023-10-12 22:38:41,023][44958] Updated weights for policy 0, policy_version 62500 (0.0008) [2023-10-12 22:38:41,411][44958] Updated weights for policy 0, policy_version 62510 (0.0008) [2023-10-12 22:38:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128319488. Throughput: 0: 1640.5, 1: 1660.2. Samples: 32094232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:41,443][43579] Avg episode reward: [(0, '279.130'), (1, '280.310')] [2023-10-12 22:38:41,738][44959] Updated weights for policy 1, policy_version 62820 (0.0009) [2023-10-12 22:38:41,793][44958] Updated weights for policy 0, policy_version 62520 (0.0007) [2023-10-12 22:38:42,091][44959] Updated weights for policy 1, policy_version 62830 (0.0010) [2023-10-12 22:38:42,467][44959] Updated weights for policy 1, policy_version 62840 (0.0010) [2023-10-12 22:38:45,902][44958] Updated weights for policy 0, policy_version 62530 (0.0009) [2023-10-12 22:38:46,284][44958] Updated weights for policy 0, policy_version 62540 (0.0009) [2023-10-12 22:38:46,398][44959] Updated weights for policy 1, policy_version 62850 (0.0009) [2023-10-12 22:38:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128385024. Throughput: 0: 1640.7, 1: 1677.4. Samples: 32114610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:46,443][43579] Avg episode reward: [(0, '281.460'), (1, '280.320')] [2023-10-12 22:38:46,647][44958] Updated weights for policy 0, policy_version 62550 (0.0009) [2023-10-12 22:38:46,817][44959] Updated weights for policy 1, policy_version 62860 (0.0007) [2023-10-12 22:38:47,022][44958] Updated weights for policy 0, policy_version 62560 (0.0010) [2023-10-12 22:38:47,187][44959] Updated weights for policy 1, policy_version 62870 (0.0008) [2023-10-12 22:38:47,549][44959] Updated weights for policy 1, policy_version 62880 (0.0009) [2023-10-12 22:38:51,235][44958] Updated weights for policy 0, policy_version 62570 (0.0008) [2023-10-12 22:38:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128450560. Throughput: 0: 1643.7, 1: 1670.2. Samples: 32123554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:51,443][43579] Avg episode reward: [(0, '279.690'), (1, '277.820')] [2023-10-12 22:38:51,595][44958] Updated weights for policy 0, policy_version 62580 (0.0007) [2023-10-12 22:38:51,793][44959] Updated weights for policy 1, policy_version 62890 (0.0008) [2023-10-12 22:38:51,966][44958] Updated weights for policy 0, policy_version 62590 (0.0007) [2023-10-12 22:38:52,163][44959] Updated weights for policy 1, policy_version 62900 (0.0009) [2023-10-12 22:38:52,537][44959] Updated weights for policy 1, policy_version 62910 (0.0010) [2023-10-12 22:38:56,110][44958] Updated weights for policy 0, policy_version 62600 (0.0008) [2023-10-12 22:38:56,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128516096. Throughput: 0: 1643.9, 1: 1669.5. Samples: 32143964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:38:56,443][43579] Avg episode reward: [(0, '276.750'), (1, '279.050')] [2023-10-12 22:38:56,487][44958] Updated weights for policy 0, policy_version 62610 (0.0008) [2023-10-12 22:38:56,676][44959] Updated weights for policy 1, policy_version 62920 (0.0007) [2023-10-12 22:38:56,867][44958] Updated weights for policy 0, policy_version 62620 (0.0011) [2023-10-12 22:38:57,048][44959] Updated weights for policy 1, policy_version 62930 (0.0009) [2023-10-12 22:38:57,426][44959] Updated weights for policy 1, policy_version 62940 (0.0008) [2023-10-12 22:39:01,003][44958] Updated weights for policy 0, policy_version 62630 (0.0010) [2023-10-12 22:39:01,366][44958] Updated weights for policy 0, policy_version 62640 (0.0010) [2023-10-12 22:39:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128581632. Throughput: 0: 1643.1, 1: 1667.1. Samples: 32163982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:39:01,443][43579] Avg episode reward: [(0, '275.910'), (1, '276.940')] [2023-10-12 22:39:01,501][44959] Updated weights for policy 1, policy_version 62950 (0.0008) [2023-10-12 22:39:01,736][44958] Updated weights for policy 0, policy_version 62650 (0.0008) [2023-10-12 22:39:01,856][44959] Updated weights for policy 1, policy_version 62960 (0.0008) [2023-10-12 22:39:02,225][44959] Updated weights for policy 1, policy_version 62970 (0.0008) [2023-10-12 22:39:05,799][44958] Updated weights for policy 0, policy_version 62660 (0.0007) [2023-10-12 22:39:06,127][44959] Updated weights for policy 1, policy_version 62980 (0.0008) [2023-10-12 22:39:06,169][44958] Updated weights for policy 0, policy_version 62670 (0.0009) [2023-10-12 22:39:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 128647168. Throughput: 0: 1644.1, 1: 1669.1. Samples: 32173348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:39:06,444][43579] Avg episode reward: [(0, '272.320'), (1, '277.340')] [2023-10-12 22:39:06,498][44959] Updated weights for policy 1, policy_version 62990 (0.0008) [2023-10-12 22:39:06,541][44958] Updated weights for policy 0, policy_version 62680 (0.0009) [2023-10-12 22:39:06,872][44959] Updated weights for policy 1, policy_version 63000 (0.0008) [2023-10-12 22:39:10,853][44958] Updated weights for policy 0, policy_version 62690 (0.0009) [2023-10-12 22:39:11,081][44959] Updated weights for policy 1, policy_version 63010 (0.0008) [2023-10-12 22:39:11,224][44958] Updated weights for policy 0, policy_version 62700 (0.0009) [2023-10-12 22:39:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128712704. Throughput: 0: 1646.9, 1: 1672.2. Samples: 32193940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:39:11,443][43579] Avg episode reward: [(0, '269.410'), (1, '274.390')] [2023-10-12 22:39:11,455][44959] Updated weights for policy 1, policy_version 63020 (0.0007) [2023-10-12 22:39:11,594][44958] Updated weights for policy 0, policy_version 62710 (0.0009) [2023-10-12 22:39:11,817][44959] Updated weights for policy 1, policy_version 63030 (0.0007) [2023-10-12 22:39:11,969][44958] Updated weights for policy 0, policy_version 62720 (0.0008) [2023-10-12 22:39:12,188][44959] Updated weights for policy 1, policy_version 63040 (0.0009) [2023-10-12 22:39:15,794][44958] Updated weights for policy 0, policy_version 62730 (0.0009) [2023-10-12 22:39:16,169][44958] Updated weights for policy 0, policy_version 62740 (0.0008) [2023-10-12 22:39:16,185][44959] Updated weights for policy 1, policy_version 63050 (0.0009) [2023-10-12 22:39:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128778240. Throughput: 0: 1635.5, 1: 1666.8. Samples: 32213306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:39:16,443][43579] Avg episode reward: [(0, '271.770'), (1, '272.420')] [2023-10-12 22:39:16,544][44958] Updated weights for policy 0, policy_version 62750 (0.0009) [2023-10-12 22:39:16,545][44959] Updated weights for policy 1, policy_version 63060 (0.0007) [2023-10-12 22:39:16,907][44959] Updated weights for policy 1, policy_version 63070 (0.0010) [2023-10-12 22:39:20,892][44958] Updated weights for policy 0, policy_version 62760 (0.0011) [2023-10-12 22:39:21,260][44959] Updated weights for policy 1, policy_version 63080 (0.0008) [2023-10-12 22:39:21,268][44958] Updated weights for policy 0, policy_version 62770 (0.0008) [2023-10-12 22:39:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128843776. Throughput: 0: 1642.8, 1: 1667.4. Samples: 32223042. Policy #0 lag: (min: 21.0, avg: 28.1, max: 53.0) [2023-10-12 22:39:21,443][43579] Avg episode reward: [(0, '266.470'), (1, '272.900')] [2023-10-12 22:39:21,624][44959] Updated weights for policy 1, policy_version 63090 (0.0008) [2023-10-12 22:39:21,639][44958] Updated weights for policy 0, policy_version 62780 (0.0007) [2023-10-12 22:39:21,979][44959] Updated weights for policy 1, policy_version 63100 (0.0009) [2023-10-12 22:39:25,719][44958] Updated weights for policy 0, policy_version 62790 (0.0009) [2023-10-12 22:39:26,094][44959] Updated weights for policy 1, policy_version 63110 (0.0007) [2023-10-12 22:39:26,098][44958] Updated weights for policy 0, policy_version 62800 (0.0009) [2023-10-12 22:39:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128909312. Throughput: 0: 1647.6, 1: 1665.6. Samples: 32243324. Policy #0 lag: (min: 21.0, avg: 28.1, max: 53.0) [2023-10-12 22:39:26,443][43579] Avg episode reward: [(0, '270.880'), (1, '283.420')] [2023-10-12 22:39:26,463][44959] Updated weights for policy 1, policy_version 63120 (0.0009) [2023-10-12 22:39:26,466][44958] Updated weights for policy 0, policy_version 62810 (0.0010) [2023-10-12 22:39:26,830][44959] Updated weights for policy 1, policy_version 63130 (0.0007) [2023-10-12 22:39:30,821][44958] Updated weights for policy 0, policy_version 62820 (0.0007) [2023-10-12 22:39:31,011][44959] Updated weights for policy 1, policy_version 63140 (0.0007) [2023-10-12 22:39:31,204][44958] Updated weights for policy 0, policy_version 62830 (0.0009) [2023-10-12 22:39:31,381][44959] Updated weights for policy 1, policy_version 63150 (0.0007) [2023-10-12 22:39:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 128974848. Throughput: 0: 1641.3, 1: 1652.1. Samples: 32262814. Policy #0 lag: (min: 21.0, avg: 28.1, max: 53.0) [2023-10-12 22:39:31,443][43579] Avg episode reward: [(0, '271.180'), (1, '280.720')] [2023-10-12 22:39:31,582][44958] Updated weights for policy 0, policy_version 62840 (0.0008) [2023-10-12 22:39:31,746][44959] Updated weights for policy 1, policy_version 63160 (0.0008) [2023-10-12 22:39:35,692][44958] Updated weights for policy 0, policy_version 62850 (0.0009) [2023-10-12 22:39:35,901][44959] Updated weights for policy 1, policy_version 63170 (0.0010) [2023-10-12 22:39:36,068][44958] Updated weights for policy 0, policy_version 62860 (0.0008) [2023-10-12 22:39:36,298][44959] Updated weights for policy 1, policy_version 63180 (0.0010) [2023-10-12 22:39:36,438][44958] Updated weights for policy 0, policy_version 62870 (0.0008) [2023-10-12 22:39:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 129040384. Throughput: 0: 1643.8, 1: 1660.4. Samples: 32272244. Policy #0 lag: (min: 21.0, avg: 28.1, max: 53.0) [2023-10-12 22:39:36,443][43579] Avg episode reward: [(0, '273.670'), (1, '282.650')] [2023-10-12 22:39:36,662][44959] Updated weights for policy 1, policy_version 63190 (0.0009) [2023-10-12 22:39:36,803][44958] Updated weights for policy 0, policy_version 62880 (0.0009) [2023-10-12 22:39:37,035][44959] Updated weights for policy 1, policy_version 63200 (0.0008) [2023-10-12 22:39:41,066][44958] Updated weights for policy 0, policy_version 62890 (0.0008) [2023-10-12 22:39:41,297][44959] Updated weights for policy 1, policy_version 63210 (0.0008) [2023-10-12 22:39:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 129105920. Throughput: 0: 1643.6, 1: 1655.7. Samples: 32292436. Policy #0 lag: (min: 21.0, avg: 28.1, max: 53.0) [2023-10-12 22:39:41,443][43579] Avg episode reward: [(0, '274.280'), (1, '283.030')] [2023-10-12 22:39:41,444][44958] Updated weights for policy 0, policy_version 62900 (0.0008) [2023-10-12 22:39:41,670][44959] Updated weights for policy 1, policy_version 63220 (0.0009) [2023-10-12 22:39:41,815][44958] Updated weights for policy 0, policy_version 62910 (0.0010) [2023-10-12 22:39:42,043][44959] Updated weights for policy 1, policy_version 63230 (0.0007) [2023-10-12 22:39:45,852][44958] Updated weights for policy 0, policy_version 62920 (0.0009) [2023-10-12 22:39:46,099][44959] Updated weights for policy 1, policy_version 63240 (0.0008) [2023-10-12 22:39:46,226][44958] Updated weights for policy 0, policy_version 62930 (0.0011) [2023-10-12 22:39:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 129171456. Throughput: 0: 1637.6, 1: 1650.5. Samples: 32311948. Policy #0 lag: (min: 21.0, avg: 28.1, max: 53.0) [2023-10-12 22:39:46,443][43579] Avg episode reward: [(0, '274.540'), (1, '283.780')] [2023-10-12 22:39:46,464][44959] Updated weights for policy 1, policy_version 63250 (0.0010) [2023-10-12 22:39:46,600][44958] Updated weights for policy 0, policy_version 62940 (0.0009) [2023-10-12 22:39:46,748][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000062944_64454656.pth... [2023-10-12 22:39:46,777][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000061408_62881792.pth [2023-10-12 22:39:46,838][44959] Updated weights for policy 1, policy_version 63260 (0.0008) [2023-10-12 22:39:46,977][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000063264_64782336.pth... [2023-10-12 22:39:47,013][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000061696_63176704.pth [2023-10-12 22:39:50,798][44958] Updated weights for policy 0, policy_version 62950 (0.0009) [2023-10-12 22:39:51,040][44959] Updated weights for policy 1, policy_version 63270 (0.0008) [2023-10-12 22:39:51,166][44958] Updated weights for policy 0, policy_version 62960 (0.0008) [2023-10-12 22:39:51,405][44959] Updated weights for policy 1, policy_version 63280 (0.0009) [2023-10-12 22:39:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 129236992. Throughput: 0: 1637.4, 1: 1650.4. Samples: 32321300. Policy #0 lag: (min: 21.0, avg: 28.1, max: 53.0) [2023-10-12 22:39:51,443][43579] Avg episode reward: [(0, '276.070'), (1, '277.420')] [2023-10-12 22:39:51,536][44958] Updated weights for policy 0, policy_version 62970 (0.0007) [2023-10-12 22:39:51,777][44959] Updated weights for policy 1, policy_version 63290 (0.0007) [2023-10-12 22:39:55,871][44958] Updated weights for policy 0, policy_version 62980 (0.0008) [2023-10-12 22:39:55,972][44959] Updated weights for policy 1, policy_version 63300 (0.0008) [2023-10-12 22:39:56,238][44958] Updated weights for policy 0, policy_version 62990 (0.0008) [2023-10-12 22:39:56,344][44959] Updated weights for policy 1, policy_version 63310 (0.0009) [2023-10-12 22:39:56,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 129302528. Throughput: 0: 1632.7, 1: 1642.1. Samples: 32341304. Policy #0 lag: (min: 21.0, avg: 28.1, max: 53.0) [2023-10-12 22:39:56,444][43579] Avg episode reward: [(0, '278.380'), (1, '279.520')] [2023-10-12 22:39:56,607][44958] Updated weights for policy 0, policy_version 63000 (0.0009) [2023-10-12 22:39:56,704][44959] Updated weights for policy 1, policy_version 63320 (0.0007) [2023-10-12 22:40:00,674][44958] Updated weights for policy 0, policy_version 63010 (0.0007) [2023-10-12 22:40:00,702][44959] Updated weights for policy 1, policy_version 63330 (0.0007) [2023-10-12 22:40:01,053][44958] Updated weights for policy 0, policy_version 63020 (0.0008) [2023-10-12 22:40:01,066][44959] Updated weights for policy 1, policy_version 63340 (0.0007) [2023-10-12 22:40:01,431][44959] Updated weights for policy 1, policy_version 63350 (0.0007) [2023-10-12 22:40:01,433][44958] Updated weights for policy 0, policy_version 63030 (0.0008) [2023-10-12 22:40:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 129368064. Throughput: 0: 1635.7, 1: 1642.3. Samples: 32360814. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 22:40:01,444][43579] Avg episode reward: [(0, '280.010'), (1, '278.410')] [2023-10-12 22:40:01,803][44958] Updated weights for policy 0, policy_version 63040 (0.0007) [2023-10-12 22:40:01,805][44959] Updated weights for policy 1, policy_version 63360 (0.0007) [2023-10-12 22:40:05,922][44959] Updated weights for policy 1, policy_version 63370 (0.0007) [2023-10-12 22:40:06,194][44958] Updated weights for policy 0, policy_version 63050 (0.0008) [2023-10-12 22:40:06,299][44959] Updated weights for policy 1, policy_version 63380 (0.0008) [2023-10-12 22:40:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 129433600. Throughput: 0: 1625.6, 1: 1648.5. Samples: 32370378. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 22:40:06,443][43579] Avg episode reward: [(0, '285.800'), (1, '275.750')] [2023-10-12 22:40:06,565][44958] Updated weights for policy 0, policy_version 63060 (0.0009) [2023-10-12 22:40:06,669][44959] Updated weights for policy 1, policy_version 63390 (0.0008) [2023-10-12 22:40:06,945][44958] Updated weights for policy 0, policy_version 63070 (0.0008) [2023-10-12 22:40:10,708][44959] Updated weights for policy 1, policy_version 63400 (0.0008) [2023-10-12 22:40:11,075][44959] Updated weights for policy 1, policy_version 63410 (0.0007) [2023-10-12 22:40:11,293][44958] Updated weights for policy 0, policy_version 63080 (0.0008) [2023-10-12 22:40:11,440][44959] Updated weights for policy 1, policy_version 63420 (0.0010) [2023-10-12 22:40:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 129499136. Throughput: 0: 1628.5, 1: 1649.2. Samples: 32390822. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 22:40:11,444][43579] Avg episode reward: [(0, '281.900'), (1, '274.190')] [2023-10-12 22:40:11,670][44958] Updated weights for policy 0, policy_version 63090 (0.0008) [2023-10-12 22:40:12,050][44958] Updated weights for policy 0, policy_version 63100 (0.0007) [2023-10-12 22:40:15,844][44959] Updated weights for policy 1, policy_version 63430 (0.0008) [2023-10-12 22:40:16,053][44958] Updated weights for policy 0, policy_version 63110 (0.0008) [2023-10-12 22:40:16,221][44959] Updated weights for policy 1, policy_version 63440 (0.0008) [2023-10-12 22:40:16,430][44958] Updated weights for policy 0, policy_version 63120 (0.0007) [2023-10-12 22:40:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 129564672. Throughput: 0: 1630.4, 1: 1641.3. Samples: 32410040. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 22:40:16,443][43579] Avg episode reward: [(0, '280.620'), (1, '273.900')] [2023-10-12 22:40:16,585][44959] Updated weights for policy 1, policy_version 63450 (0.0008) [2023-10-12 22:40:16,803][44958] Updated weights for policy 0, policy_version 63130 (0.0007) [2023-10-12 22:40:20,951][44959] Updated weights for policy 1, policy_version 63460 (0.0008) [2023-10-12 22:40:20,983][44958] Updated weights for policy 0, policy_version 63140 (0.0009) [2023-10-12 22:40:21,355][44959] Updated weights for policy 1, policy_version 63470 (0.0009) [2023-10-12 22:40:21,355][44958] Updated weights for policy 0, policy_version 63150 (0.0010) [2023-10-12 22:40:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 129630208. Throughput: 0: 1631.4, 1: 1648.5. Samples: 32419840. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 22:40:21,443][43579] Avg episode reward: [(0, '277.070'), (1, '274.660')] [2023-10-12 22:40:21,720][44959] Updated weights for policy 1, policy_version 63480 (0.0009) [2023-10-12 22:40:21,732][44958] Updated weights for policy 0, policy_version 63160 (0.0007) [2023-10-12 22:40:25,770][44959] Updated weights for policy 1, policy_version 63490 (0.0008) [2023-10-12 22:40:25,909][44958] Updated weights for policy 0, policy_version 63170 (0.0008) [2023-10-12 22:40:26,140][44959] Updated weights for policy 1, policy_version 63500 (0.0009) [2023-10-12 22:40:26,278][44958] Updated weights for policy 0, policy_version 63180 (0.0008) [2023-10-12 22:40:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 129695744. Throughput: 0: 1630.2, 1: 1646.0. Samples: 32439866. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 22:40:26,444][43579] Avg episode reward: [(0, '273.790'), (1, '275.410')] [2023-10-12 22:40:26,501][44959] Updated weights for policy 1, policy_version 63510 (0.0009) [2023-10-12 22:40:26,656][44958] Updated weights for policy 0, policy_version 63190 (0.0009) [2023-10-12 22:40:26,872][44959] Updated weights for policy 1, policy_version 63520 (0.0009) [2023-10-12 22:40:27,029][44958] Updated weights for policy 0, policy_version 63200 (0.0009) [2023-10-12 22:40:30,984][44959] Updated weights for policy 1, policy_version 63530 (0.0007) [2023-10-12 22:40:31,182][44958] Updated weights for policy 0, policy_version 63210 (0.0008) [2023-10-12 22:40:31,352][44959] Updated weights for policy 1, policy_version 63540 (0.0007) [2023-10-12 22:40:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 129761280. Throughput: 0: 1632.1, 1: 1638.4. Samples: 32459124. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 22:40:31,444][43579] Avg episode reward: [(0, '272.000'), (1, '272.580')] [2023-10-12 22:40:31,560][44958] Updated weights for policy 0, policy_version 63220 (0.0008) [2023-10-12 22:40:31,709][44959] Updated weights for policy 1, policy_version 63550 (0.0008) [2023-10-12 22:40:31,930][44958] Updated weights for policy 0, policy_version 63230 (0.0008) [2023-10-12 22:40:35,842][44959] Updated weights for policy 1, policy_version 63560 (0.0008) [2023-10-12 22:40:36,162][44958] Updated weights for policy 0, policy_version 63240 (0.0009) [2023-10-12 22:40:36,210][44959] Updated weights for policy 1, policy_version 63570 (0.0007) [2023-10-12 22:40:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 129826816. Throughput: 0: 1629.6, 1: 1646.0. Samples: 32468706. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 22:40:36,443][43579] Avg episode reward: [(0, '270.860'), (1, '276.550')] [2023-10-12 22:40:36,528][44958] Updated weights for policy 0, policy_version 63250 (0.0009) [2023-10-12 22:40:36,585][44959] Updated weights for policy 1, policy_version 63580 (0.0007) [2023-10-12 22:40:36,887][44958] Updated weights for policy 0, policy_version 63260 (0.0007) [2023-10-12 22:40:40,802][44959] Updated weights for policy 1, policy_version 63590 (0.0010) [2023-10-12 22:40:41,019][44958] Updated weights for policy 0, policy_version 63270 (0.0009) [2023-10-12 22:40:41,165][44959] Updated weights for policy 1, policy_version 63600 (0.0010) [2023-10-12 22:40:41,397][44958] Updated weights for policy 0, policy_version 63280 (0.0009) [2023-10-12 22:40:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 129892352. Throughput: 0: 1633.8, 1: 1652.1. Samples: 32489168. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-12 22:40:41,443][43579] Avg episode reward: [(0, '271.250'), (1, '276.120')] [2023-10-12 22:40:41,538][44959] Updated weights for policy 1, policy_version 63610 (0.0008) [2023-10-12 22:40:41,759][44958] Updated weights for policy 0, policy_version 63290 (0.0007) [2023-10-12 22:40:45,577][44959] Updated weights for policy 1, policy_version 63620 (0.0008) [2023-10-12 22:40:45,888][44958] Updated weights for policy 0, policy_version 63300 (0.0009) [2023-10-12 22:40:45,948][44959] Updated weights for policy 1, policy_version 63630 (0.0008) [2023-10-12 22:40:46,265][44958] Updated weights for policy 0, policy_version 63310 (0.0007) [2023-10-12 22:40:46,313][44959] Updated weights for policy 1, policy_version 63640 (0.0009) [2023-10-12 22:40:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 129957888. Throughput: 0: 1639.2, 1: 1642.3. Samples: 32508480. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-12 22:40:46,443][43579] Avg episode reward: [(0, '272.080'), (1, '277.160')] [2023-10-12 22:40:46,635][44958] Updated weights for policy 0, policy_version 63320 (0.0007) [2023-10-12 22:40:50,534][44959] Updated weights for policy 1, policy_version 63650 (0.0009) [2023-10-12 22:40:50,767][44958] Updated weights for policy 0, policy_version 63330 (0.0007) [2023-10-12 22:40:50,904][44959] Updated weights for policy 1, policy_version 63660 (0.0007) [2023-10-12 22:40:51,133][44958] Updated weights for policy 0, policy_version 63340 (0.0007) [2023-10-12 22:40:51,274][44959] Updated weights for policy 1, policy_version 63670 (0.0007) [2023-10-12 22:40:51,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 130023424. Throughput: 0: 1639.9, 1: 1645.3. Samples: 32518214. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-12 22:40:51,444][43579] Avg episode reward: [(0, '276.530'), (1, '276.850')] [2023-10-12 22:40:51,509][44958] Updated weights for policy 0, policy_version 63350 (0.0010) [2023-10-12 22:40:51,647][44959] Updated weights for policy 1, policy_version 63680 (0.0008) [2023-10-12 22:40:51,886][44958] Updated weights for policy 0, policy_version 63360 (0.0008) [2023-10-12 22:40:55,629][44959] Updated weights for policy 1, policy_version 63690 (0.0009) [2023-10-12 22:40:55,994][44959] Updated weights for policy 1, policy_version 63700 (0.0008) [2023-10-12 22:40:56,179][44958] Updated weights for policy 0, policy_version 63370 (0.0008) [2023-10-12 22:40:56,361][44959] Updated weights for policy 1, policy_version 63710 (0.0009) [2023-10-12 22:40:56,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 130121728. Throughput: 0: 1641.9, 1: 1647.6. Samples: 32538848. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-12 22:40:56,443][43579] Avg episode reward: [(0, '276.070'), (1, '276.750')] [2023-10-12 22:40:56,557][44958] Updated weights for policy 0, policy_version 63380 (0.0009) [2023-10-12 22:40:56,934][44958] Updated weights for policy 0, policy_version 63390 (0.0008) [2023-10-12 22:41:00,622][44959] Updated weights for policy 1, policy_version 63720 (0.0009) [2023-10-12 22:41:00,991][44958] Updated weights for policy 0, policy_version 63400 (0.0009) [2023-10-12 22:41:00,997][44959] Updated weights for policy 1, policy_version 63730 (0.0007) [2023-10-12 22:41:01,360][44959] Updated weights for policy 1, policy_version 63740 (0.0007) [2023-10-12 22:41:01,374][44958] Updated weights for policy 0, policy_version 63410 (0.0008) [2023-10-12 22:41:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 130154496. Throughput: 0: 1641.3, 1: 1640.9. Samples: 32557738. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-12 22:41:01,444][43579] Avg episode reward: [(0, '273.070'), (1, '282.010')] [2023-10-12 22:41:01,746][44958] Updated weights for policy 0, policy_version 63420 (0.0010) [2023-10-12 22:41:05,557][44959] Updated weights for policy 1, policy_version 63750 (0.0007) [2023-10-12 22:41:05,955][44959] Updated weights for policy 1, policy_version 63760 (0.0007) [2023-10-12 22:41:05,979][44958] Updated weights for policy 0, policy_version 63430 (0.0008) [2023-10-12 22:41:06,325][44959] Updated weights for policy 1, policy_version 63770 (0.0009) [2023-10-12 22:41:06,349][44958] Updated weights for policy 0, policy_version 63440 (0.0009) [2023-10-12 22:41:06,442][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 130220032. Throughput: 0: 1639.7, 1: 1647.1. Samples: 32567748. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-12 22:41:06,443][43579] Avg episode reward: [(0, '274.820'), (1, '277.700')] [2023-10-12 22:41:06,724][44958] Updated weights for policy 0, policy_version 63450 (0.0008) [2023-10-12 22:41:10,494][44959] Updated weights for policy 1, policy_version 63780 (0.0009) [2023-10-12 22:41:10,856][44959] Updated weights for policy 1, policy_version 63790 (0.0009) [2023-10-12 22:41:10,977][44958] Updated weights for policy 0, policy_version 63460 (0.0008) [2023-10-12 22:41:11,230][44959] Updated weights for policy 1, policy_version 63800 (0.0010) [2023-10-12 22:41:11,344][44958] Updated weights for policy 0, policy_version 63470 (0.0007) [2023-10-12 22:41:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 130285568. Throughput: 0: 1639.0, 1: 1653.2. Samples: 32588018. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-12 22:41:11,444][43579] Avg episode reward: [(0, '271.910'), (1, '273.500')] [2023-10-12 22:41:11,720][44958] Updated weights for policy 0, policy_version 63480 (0.0009) [2023-10-12 22:41:15,386][44959] Updated weights for policy 1, policy_version 63810 (0.0009) [2023-10-12 22:41:15,757][44959] Updated weights for policy 1, policy_version 63820 (0.0009) [2023-10-12 22:41:15,889][44958] Updated weights for policy 0, policy_version 63490 (0.0009) [2023-10-12 22:41:16,122][44959] Updated weights for policy 1, policy_version 63830 (0.0007) [2023-10-12 22:41:16,262][44958] Updated weights for policy 0, policy_version 63500 (0.0009) [2023-10-12 22:41:16,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 130351104. Throughput: 0: 1642.7, 1: 1647.5. Samples: 32607182. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-12 22:41:16,443][43579] Avg episode reward: [(0, '273.800'), (1, '270.160')] [2023-10-12 22:41:16,483][44959] Updated weights for policy 1, policy_version 63840 (0.0007) [2023-10-12 22:41:16,627][44958] Updated weights for policy 0, policy_version 63510 (0.0008) [2023-10-12 22:41:17,003][44958] Updated weights for policy 0, policy_version 63520 (0.0009) [2023-10-12 22:41:20,704][44959] Updated weights for policy 1, policy_version 63850 (0.0007) [2023-10-12 22:41:21,065][44959] Updated weights for policy 1, policy_version 63860 (0.0008) [2023-10-12 22:41:21,202][44958] Updated weights for policy 0, policy_version 63530 (0.0008) [2023-10-12 22:41:21,434][44959] Updated weights for policy 1, policy_version 63870 (0.0008) [2023-10-12 22:41:21,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 130416640. Throughput: 0: 1642.0, 1: 1656.6. Samples: 32617146. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-12 22:41:21,443][43579] Avg episode reward: [(0, '273.110'), (1, '271.550')] [2023-10-12 22:41:21,578][44958] Updated weights for policy 0, policy_version 63540 (0.0010) [2023-10-12 22:41:21,951][44958] Updated weights for policy 0, policy_version 63550 (0.0011) [2023-10-12 22:41:25,556][44959] Updated weights for policy 1, policy_version 63880 (0.0007) [2023-10-12 22:41:25,921][44959] Updated weights for policy 1, policy_version 63890 (0.0007) [2023-10-12 22:41:26,096][44958] Updated weights for policy 0, policy_version 63560 (0.0009) [2023-10-12 22:41:26,285][44959] Updated weights for policy 1, policy_version 63900 (0.0009) [2023-10-12 22:41:26,442][43579] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 130514944. Throughput: 0: 1639.9, 1: 1652.8. Samples: 32637340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:41:26,443][43579] Avg episode reward: [(0, '275.950'), (1, '268.090')] [2023-10-12 22:41:26,472][44958] Updated weights for policy 0, policy_version 63570 (0.0009) [2023-10-12 22:41:26,839][44958] Updated weights for policy 0, policy_version 63580 (0.0010) [2023-10-12 22:41:30,490][44959] Updated weights for policy 1, policy_version 63910 (0.0010) [2023-10-12 22:41:30,860][44959] Updated weights for policy 1, policy_version 63920 (0.0008) [2023-10-12 22:41:31,051][44958] Updated weights for policy 0, policy_version 63590 (0.0008) [2023-10-12 22:41:31,228][44959] Updated weights for policy 1, policy_version 63930 (0.0007) [2023-10-12 22:41:31,426][44958] Updated weights for policy 0, policy_version 63600 (0.0009) [2023-10-12 22:41:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 130547712. Throughput: 0: 1636.3, 1: 1648.6. Samples: 32656302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:41:31,443][43579] Avg episode reward: [(0, '278.130'), (1, '267.320')] [2023-10-12 22:41:31,795][44958] Updated weights for policy 0, policy_version 63610 (0.0010) [2023-10-12 22:41:35,348][44959] Updated weights for policy 1, policy_version 63940 (0.0009) [2023-10-12 22:41:35,713][44959] Updated weights for policy 1, policy_version 63950 (0.0007) [2023-10-12 22:41:35,906][44958] Updated weights for policy 0, policy_version 63620 (0.0009) [2023-10-12 22:41:36,083][44959] Updated weights for policy 1, policy_version 63960 (0.0007) [2023-10-12 22:41:36,285][44958] Updated weights for policy 0, policy_version 63630 (0.0009) [2023-10-12 22:41:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 130646016. Throughput: 0: 1638.9, 1: 1654.9. Samples: 32666436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:41:36,443][43579] Avg episode reward: [(0, '279.960'), (1, '271.410')] [2023-10-12 22:41:36,654][44958] Updated weights for policy 0, policy_version 63640 (0.0007) [2023-10-12 22:41:40,192][44959] Updated weights for policy 1, policy_version 63970 (0.0008) [2023-10-12 22:41:40,560][44959] Updated weights for policy 1, policy_version 63980 (0.0008) [2023-10-12 22:41:40,931][44958] Updated weights for policy 0, policy_version 63650 (0.0008) [2023-10-12 22:41:40,932][44959] Updated weights for policy 1, policy_version 63990 (0.0007) [2023-10-12 22:41:41,301][44959] Updated weights for policy 1, policy_version 64000 (0.0010) [2023-10-12 22:41:41,327][44958] Updated weights for policy 0, policy_version 63660 (0.0008) [2023-10-12 22:41:41,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 130711552. Throughput: 0: 1631.2, 1: 1650.4. Samples: 32686524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:41:41,443][43579] Avg episode reward: [(0, '280.630'), (1, '273.200')] [2023-10-12 22:41:41,696][44958] Updated weights for policy 0, policy_version 63670 (0.0010) [2023-10-12 22:41:42,067][44958] Updated weights for policy 0, policy_version 63680 (0.0011) [2023-10-12 22:41:45,334][44959] Updated weights for policy 1, policy_version 64010 (0.0010) [2023-10-12 22:41:45,699][44959] Updated weights for policy 1, policy_version 64020 (0.0009) [2023-10-12 22:41:46,064][44959] Updated weights for policy 1, policy_version 64030 (0.0008) [2023-10-12 22:41:46,314][44958] Updated weights for policy 0, policy_version 63690 (0.0009) [2023-10-12 22:41:46,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 130777088. Throughput: 0: 1639.6, 1: 1647.6. Samples: 32705662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:41:46,444][43579] Avg episode reward: [(0, '277.800'), (1, '274.810')] [2023-10-12 22:41:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000064032_65568768.pth... [2023-10-12 22:41:46,484][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000062464_63963136.pth [2023-10-12 22:41:46,686][44958] Updated weights for policy 0, policy_version 63700 (0.0009) [2023-10-12 22:41:47,059][44958] Updated weights for policy 0, policy_version 63710 (0.0009) [2023-10-12 22:41:47,126][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000063712_65241088.pth... [2023-10-12 22:41:47,163][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000062176_63668224.pth [2023-10-12 22:41:50,371][44959] Updated weights for policy 1, policy_version 64040 (0.0009) [2023-10-12 22:41:50,745][44959] Updated weights for policy 1, policy_version 64050 (0.0010) [2023-10-12 22:41:51,110][44959] Updated weights for policy 1, policy_version 64060 (0.0009) [2023-10-12 22:41:51,307][44958] Updated weights for policy 0, policy_version 63720 (0.0010) [2023-10-12 22:41:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 130842624. Throughput: 0: 1630.9, 1: 1656.4. Samples: 32715678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:41:51,443][43579] Avg episode reward: [(0, '281.880'), (1, '272.030')] [2023-10-12 22:41:51,683][44958] Updated weights for policy 0, policy_version 63730 (0.0009) [2023-10-12 22:41:52,051][44958] Updated weights for policy 0, policy_version 63740 (0.0009) [2023-10-12 22:41:55,277][44959] Updated weights for policy 1, policy_version 64070 (0.0008) [2023-10-12 22:41:55,637][44959] Updated weights for policy 1, policy_version 64080 (0.0007) [2023-10-12 22:41:56,007][44959] Updated weights for policy 1, policy_version 64090 (0.0009) [2023-10-12 22:41:56,167][44958] Updated weights for policy 0, policy_version 63750 (0.0008) [2023-10-12 22:41:56,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 130908160. Throughput: 0: 1630.4, 1: 1651.5. Samples: 32735700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:41:56,443][43579] Avg episode reward: [(0, '286.250'), (1, '274.950')] [2023-10-12 22:41:56,536][44958] Updated weights for policy 0, policy_version 63760 (0.0009) [2023-10-12 22:41:56,903][44958] Updated weights for policy 0, policy_version 63770 (0.0009) [2023-10-12 22:42:00,247][44959] Updated weights for policy 1, policy_version 64100 (0.0009) [2023-10-12 22:42:00,614][44959] Updated weights for policy 1, policy_version 64110 (0.0008) [2023-10-12 22:42:00,982][44959] Updated weights for policy 1, policy_version 64120 (0.0010) [2023-10-12 22:42:01,120][44958] Updated weights for policy 0, policy_version 63780 (0.0009) [2023-10-12 22:42:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 130973696. Throughput: 0: 1633.6, 1: 1643.6. Samples: 32754656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:42:01,444][43579] Avg episode reward: [(0, '277.770'), (1, '276.140')] [2023-10-12 22:42:01,482][44958] Updated weights for policy 0, policy_version 63790 (0.0008) [2023-10-12 22:42:01,859][44958] Updated weights for policy 0, policy_version 63800 (0.0008) [2023-10-12 22:42:05,142][44959] Updated weights for policy 1, policy_version 64130 (0.0008) [2023-10-12 22:42:05,513][44959] Updated weights for policy 1, policy_version 64140 (0.0010) [2023-10-12 22:42:05,882][44959] Updated weights for policy 1, policy_version 64150 (0.0008) [2023-10-12 22:42:05,945][44958] Updated weights for policy 0, policy_version 63810 (0.0010) [2023-10-12 22:42:06,254][44959] Updated weights for policy 1, policy_version 64160 (0.0008) [2023-10-12 22:42:06,307][44958] Updated weights for policy 0, policy_version 63820 (0.0009) [2023-10-12 22:42:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 131039232. Throughput: 0: 1629.7, 1: 1646.9. Samples: 32764594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:42:06,443][43579] Avg episode reward: [(0, '281.920'), (1, '276.600')] [2023-10-12 22:42:06,678][44958] Updated weights for policy 0, policy_version 63830 (0.0009) [2023-10-12 22:42:07,050][44958] Updated weights for policy 0, policy_version 63840 (0.0010) [2023-10-12 22:42:10,419][44959] Updated weights for policy 1, policy_version 64170 (0.0007) [2023-10-12 22:42:10,790][44959] Updated weights for policy 1, policy_version 64180 (0.0008) [2023-10-12 22:42:11,148][44959] Updated weights for policy 1, policy_version 64190 (0.0007) [2023-10-12 22:42:11,262][44958] Updated weights for policy 0, policy_version 63850 (0.0007) [2023-10-12 22:42:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 131104768. Throughput: 0: 1632.5, 1: 1649.3. Samples: 32785022. Policy #0 lag: (min: 17.0, avg: 28.0, max: 49.0) [2023-10-12 22:42:11,444][43579] Avg episode reward: [(0, '283.440'), (1, '278.000')] [2023-10-12 22:42:11,629][44958] Updated weights for policy 0, policy_version 63860 (0.0008) [2023-10-12 22:42:12,015][44958] Updated weights for policy 0, policy_version 63870 (0.0008) [2023-10-12 22:42:15,251][44959] Updated weights for policy 1, policy_version 64200 (0.0008) [2023-10-12 22:42:15,614][44959] Updated weights for policy 1, policy_version 64210 (0.0010) [2023-10-12 22:42:15,991][44959] Updated weights for policy 1, policy_version 64220 (0.0010) [2023-10-12 22:42:16,200][44958] Updated weights for policy 0, policy_version 63880 (0.0010) [2023-10-12 22:42:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 131170304. Throughput: 0: 1639.0, 1: 1639.3. Samples: 32803828. Policy #0 lag: (min: 17.0, avg: 28.0, max: 49.0) [2023-10-12 22:42:16,444][43579] Avg episode reward: [(0, '283.500'), (1, '281.110')] [2023-10-12 22:42:16,578][44958] Updated weights for policy 0, policy_version 63890 (0.0009) [2023-10-12 22:42:16,955][44958] Updated weights for policy 0, policy_version 63900 (0.0009) [2023-10-12 22:42:20,190][44959] Updated weights for policy 1, policy_version 64230 (0.0008) [2023-10-12 22:42:20,551][44959] Updated weights for policy 1, policy_version 64240 (0.0010) [2023-10-12 22:42:20,914][44959] Updated weights for policy 1, policy_version 64250 (0.0010) [2023-10-12 22:42:21,260][44958] Updated weights for policy 0, policy_version 63910 (0.0009) [2023-10-12 22:42:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 131235840. Throughput: 0: 1631.1, 1: 1644.5. Samples: 32813838. Policy #0 lag: (min: 17.0, avg: 28.0, max: 49.0) [2023-10-12 22:42:21,443][43579] Avg episode reward: [(0, '280.550'), (1, '282.950')] [2023-10-12 22:42:21,628][44958] Updated weights for policy 0, policy_version 63920 (0.0010) [2023-10-12 22:42:21,997][44958] Updated weights for policy 0, policy_version 63930 (0.0009) [2023-10-12 22:42:25,072][44959] Updated weights for policy 1, policy_version 64260 (0.0009) [2023-10-12 22:42:25,447][44959] Updated weights for policy 1, policy_version 64270 (0.0007) [2023-10-12 22:42:25,816][44959] Updated weights for policy 1, policy_version 64280 (0.0008) [2023-10-12 22:42:26,197][44958] Updated weights for policy 0, policy_version 63940 (0.0009) [2023-10-12 22:42:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131301376. Throughput: 0: 1637.1, 1: 1647.0. Samples: 32834308. Policy #0 lag: (min: 17.0, avg: 28.0, max: 49.0) [2023-10-12 22:42:26,443][43579] Avg episode reward: [(0, '280.090'), (1, '279.720')] [2023-10-12 22:42:26,589][44958] Updated weights for policy 0, policy_version 63950 (0.0010) [2023-10-12 22:42:26,960][44958] Updated weights for policy 0, policy_version 63960 (0.0007) [2023-10-12 22:42:29,864][44959] Updated weights for policy 1, policy_version 64290 (0.0008) [2023-10-12 22:42:30,226][44959] Updated weights for policy 1, policy_version 64300 (0.0009) [2023-10-12 22:42:30,588][44959] Updated weights for policy 1, policy_version 64310 (0.0007) [2023-10-12 22:42:30,949][44959] Updated weights for policy 1, policy_version 64320 (0.0007) [2023-10-12 22:42:31,187][44958] Updated weights for policy 0, policy_version 63970 (0.0008) [2023-10-12 22:42:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 131366912. Throughput: 0: 1636.5, 1: 1645.4. Samples: 32853348. Policy #0 lag: (min: 17.0, avg: 28.0, max: 49.0) [2023-10-12 22:42:31,444][43579] Avg episode reward: [(0, '282.600'), (1, '282.480')] [2023-10-12 22:42:31,565][44958] Updated weights for policy 0, policy_version 63980 (0.0009) [2023-10-12 22:42:31,939][44958] Updated weights for policy 0, policy_version 63990 (0.0009) [2023-10-12 22:42:32,307][44958] Updated weights for policy 0, policy_version 64000 (0.0009) [2023-10-12 22:42:34,972][44959] Updated weights for policy 1, policy_version 64330 (0.0008) [2023-10-12 22:42:35,343][44959] Updated weights for policy 1, policy_version 64340 (0.0007) [2023-10-12 22:42:35,714][44959] Updated weights for policy 1, policy_version 64350 (0.0009) [2023-10-12 22:42:36,316][44958] Updated weights for policy 0, policy_version 64010 (0.0010) [2023-10-12 22:42:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131432448. Throughput: 0: 1631.7, 1: 1650.5. Samples: 32863380. Policy #0 lag: (min: 17.0, avg: 28.0, max: 49.0) [2023-10-12 22:42:36,443][43579] Avg episode reward: [(0, '279.590'), (1, '277.640')] [2023-10-12 22:42:36,692][44958] Updated weights for policy 0, policy_version 64020 (0.0009) [2023-10-12 22:42:37,070][44958] Updated weights for policy 0, policy_version 64030 (0.0010) [2023-10-12 22:42:39,954][44959] Updated weights for policy 1, policy_version 64360 (0.0008) [2023-10-12 22:42:40,321][44959] Updated weights for policy 1, policy_version 64370 (0.0008) [2023-10-12 22:42:40,685][44959] Updated weights for policy 1, policy_version 64380 (0.0007) [2023-10-12 22:42:41,295][44958] Updated weights for policy 0, policy_version 64040 (0.0008) [2023-10-12 22:42:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131497984. Throughput: 0: 1632.9, 1: 1647.5. Samples: 32883316. Policy #0 lag: (min: 17.0, avg: 28.0, max: 49.0) [2023-10-12 22:42:41,443][43579] Avg episode reward: [(0, '279.060'), (1, '277.020')] [2023-10-12 22:42:41,672][44958] Updated weights for policy 0, policy_version 64050 (0.0008) [2023-10-12 22:42:42,049][44958] Updated weights for policy 0, policy_version 64060 (0.0007) [2023-10-12 22:42:44,701][44959] Updated weights for policy 1, policy_version 64390 (0.0007) [2023-10-12 22:42:45,075][44959] Updated weights for policy 1, policy_version 64400 (0.0007) [2023-10-12 22:42:45,446][44959] Updated weights for policy 1, policy_version 64410 (0.0007) [2023-10-12 22:42:46,285][44958] Updated weights for policy 0, policy_version 64070 (0.0009) [2023-10-12 22:42:46,442][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 131563520. Throughput: 0: 1637.2, 1: 1656.1. Samples: 32902856. Policy #0 lag: (min: 17.0, avg: 28.0, max: 49.0) [2023-10-12 22:42:46,443][43579] Avg episode reward: [(0, '281.950'), (1, '280.210')] [2023-10-12 22:42:46,664][44958] Updated weights for policy 0, policy_version 64080 (0.0011) [2023-10-12 22:42:47,033][44958] Updated weights for policy 0, policy_version 64090 (0.0011) [2023-10-12 22:42:49,567][44959] Updated weights for policy 1, policy_version 64420 (0.0010) [2023-10-12 22:42:49,946][44959] Updated weights for policy 1, policy_version 64430 (0.0009) [2023-10-12 22:42:50,317][44959] Updated weights for policy 1, policy_version 64440 (0.0009) [2023-10-12 22:42:50,989][44958] Updated weights for policy 0, policy_version 64100 (0.0008) [2023-10-12 22:42:51,362][44958] Updated weights for policy 0, policy_version 64110 (0.0007) [2023-10-12 22:42:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131629056. Throughput: 0: 1633.5, 1: 1664.0. Samples: 32912980. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 22:42:51,443][43579] Avg episode reward: [(0, '283.670'), (1, '280.800')] [2023-10-12 22:42:51,739][44958] Updated weights for policy 0, policy_version 64120 (0.0007) [2023-10-12 22:42:54,528][44959] Updated weights for policy 1, policy_version 64450 (0.0009) [2023-10-12 22:42:54,900][44959] Updated weights for policy 1, policy_version 64460 (0.0010) [2023-10-12 22:42:55,275][44959] Updated weights for policy 1, policy_version 64470 (0.0009) [2023-10-12 22:42:55,638][44959] Updated weights for policy 1, policy_version 64480 (0.0010) [2023-10-12 22:42:55,892][44958] Updated weights for policy 0, policy_version 64130 (0.0008) [2023-10-12 22:42:56,253][44958] Updated weights for policy 0, policy_version 64140 (0.0009) [2023-10-12 22:42:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131694592. Throughput: 0: 1637.6, 1: 1644.7. Samples: 32932722. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 22:42:56,443][43579] Avg episode reward: [(0, '284.880'), (1, '276.140')] [2023-10-12 22:42:56,631][44958] Updated weights for policy 0, policy_version 64150 (0.0010) [2023-10-12 22:42:56,998][44958] Updated weights for policy 0, policy_version 64160 (0.0010) [2023-10-12 22:42:59,895][44959] Updated weights for policy 1, policy_version 64490 (0.0008) [2023-10-12 22:43:00,260][44959] Updated weights for policy 1, policy_version 64500 (0.0008) [2023-10-12 22:43:00,630][44959] Updated weights for policy 1, policy_version 64510 (0.0007) [2023-10-12 22:43:01,215][44958] Updated weights for policy 0, policy_version 64170 (0.0010) [2023-10-12 22:43:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131760128. Throughput: 0: 1632.7, 1: 1658.6. Samples: 32951938. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 22:43:01,444][43579] Avg episode reward: [(0, '286.480'), (1, '274.910')] [2023-10-12 22:43:01,586][44958] Updated weights for policy 0, policy_version 64180 (0.0009) [2023-10-12 22:43:01,949][44958] Updated weights for policy 0, policy_version 64190 (0.0008) [2023-10-12 22:43:04,790][44959] Updated weights for policy 1, policy_version 64520 (0.0007) [2023-10-12 22:43:05,172][44959] Updated weights for policy 1, policy_version 64530 (0.0007) [2023-10-12 22:43:05,531][44959] Updated weights for policy 1, policy_version 64540 (0.0008) [2023-10-12 22:43:06,102][44958] Updated weights for policy 0, policy_version 64200 (0.0009) [2023-10-12 22:43:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131825664. Throughput: 0: 1638.6, 1: 1660.8. Samples: 32962312. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 22:43:06,444][43579] Avg episode reward: [(0, '284.140'), (1, '279.770')] [2023-10-12 22:43:06,463][44958] Updated weights for policy 0, policy_version 64210 (0.0011) [2023-10-12 22:43:06,845][44958] Updated weights for policy 0, policy_version 64220 (0.0010) [2023-10-12 22:43:09,613][44959] Updated weights for policy 1, policy_version 64550 (0.0008) [2023-10-12 22:43:09,983][44959] Updated weights for policy 1, policy_version 64560 (0.0011) [2023-10-12 22:43:10,345][44959] Updated weights for policy 1, policy_version 64570 (0.0009) [2023-10-12 22:43:11,227][44958] Updated weights for policy 0, policy_version 64230 (0.0009) [2023-10-12 22:43:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131891200. Throughput: 0: 1641.1, 1: 1644.0. Samples: 32982136. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 22:43:11,443][43579] Avg episode reward: [(0, '285.060'), (1, '281.440')] [2023-10-12 22:43:11,613][44958] Updated weights for policy 0, policy_version 64240 (0.0007) [2023-10-12 22:43:11,987][44958] Updated weights for policy 0, policy_version 64250 (0.0008) [2023-10-12 22:43:14,262][44959] Updated weights for policy 1, policy_version 64580 (0.0008) [2023-10-12 22:43:14,622][44959] Updated weights for policy 1, policy_version 64590 (0.0011) [2023-10-12 22:43:14,993][44959] Updated weights for policy 1, policy_version 64600 (0.0010) [2023-10-12 22:43:16,085][44958] Updated weights for policy 0, policy_version 64260 (0.0008) [2023-10-12 22:43:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 131956736. Throughput: 0: 1636.3, 1: 1659.1. Samples: 33001642. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 22:43:16,443][43579] Avg episode reward: [(0, '282.270'), (1, '277.120')] [2023-10-12 22:43:16,451][44958] Updated weights for policy 0, policy_version 64270 (0.0009) [2023-10-12 22:43:16,823][44958] Updated weights for policy 0, policy_version 64280 (0.0008) [2023-10-12 22:43:19,092][44959] Updated weights for policy 1, policy_version 64610 (0.0008) [2023-10-12 22:43:19,461][44959] Updated weights for policy 1, policy_version 64620 (0.0007) [2023-10-12 22:43:19,821][44959] Updated weights for policy 1, policy_version 64630 (0.0008) [2023-10-12 22:43:20,188][44959] Updated weights for policy 1, policy_version 64640 (0.0009) [2023-10-12 22:43:20,959][44958] Updated weights for policy 0, policy_version 64290 (0.0009) [2023-10-12 22:43:21,330][44958] Updated weights for policy 0, policy_version 64300 (0.0008) [2023-10-12 22:43:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132022272. Throughput: 0: 1645.3, 1: 1657.4. Samples: 33012000. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 22:43:21,443][43579] Avg episode reward: [(0, '282.630'), (1, '278.400')] [2023-10-12 22:43:21,691][44958] Updated weights for policy 0, policy_version 64310 (0.0008) [2023-10-12 22:43:22,066][44958] Updated weights for policy 0, policy_version 64320 (0.0009) [2023-10-12 22:43:24,437][44959] Updated weights for policy 1, policy_version 64650 (0.0008) [2023-10-12 22:43:24,809][44959] Updated weights for policy 1, policy_version 64660 (0.0007) [2023-10-12 22:43:25,175][44959] Updated weights for policy 1, policy_version 64670 (0.0007) [2023-10-12 22:43:26,297][44958] Updated weights for policy 0, policy_version 64330 (0.0007) [2023-10-12 22:43:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132087808. Throughput: 0: 1646.2, 1: 1640.9. Samples: 33031238. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 22:43:26,444][43579] Avg episode reward: [(0, '282.580'), (1, '282.590')] [2023-10-12 22:43:26,667][44958] Updated weights for policy 0, policy_version 64340 (0.0008) [2023-10-12 22:43:27,034][44958] Updated weights for policy 0, policy_version 64350 (0.0007) [2023-10-12 22:43:29,305][44959] Updated weights for policy 1, policy_version 64680 (0.0008) [2023-10-12 22:43:29,672][44959] Updated weights for policy 1, policy_version 64690 (0.0010) [2023-10-12 22:43:30,032][44959] Updated weights for policy 1, policy_version 64700 (0.0007) [2023-10-12 22:43:31,256][44958] Updated weights for policy 0, policy_version 64360 (0.0009) [2023-10-12 22:43:31,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132153344. Throughput: 0: 1640.2, 1: 1654.3. Samples: 33051112. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-12 22:43:31,444][43579] Avg episode reward: [(0, '281.070'), (1, '284.260')] [2023-10-12 22:43:31,619][44958] Updated weights for policy 0, policy_version 64370 (0.0007) [2023-10-12 22:43:31,995][44958] Updated weights for policy 0, policy_version 64380 (0.0009) [2023-10-12 22:43:34,225][44959] Updated weights for policy 1, policy_version 64710 (0.0008) [2023-10-12 22:43:34,591][44959] Updated weights for policy 1, policy_version 64720 (0.0007) [2023-10-12 22:43:34,962][44959] Updated weights for policy 1, policy_version 64730 (0.0010) [2023-10-12 22:43:36,034][44958] Updated weights for policy 0, policy_version 64390 (0.0009) [2023-10-12 22:43:36,412][44958] Updated weights for policy 0, policy_version 64400 (0.0010) [2023-10-12 22:43:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132218880. Throughput: 0: 1647.2, 1: 1648.2. Samples: 33061276. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 22:43:36,443][43579] Avg episode reward: [(0, '282.790'), (1, '282.240')] [2023-10-12 22:43:36,772][44958] Updated weights for policy 0, policy_version 64410 (0.0008) [2023-10-12 22:43:39,227][44959] Updated weights for policy 1, policy_version 64740 (0.0008) [2023-10-12 22:43:39,598][44959] Updated weights for policy 1, policy_version 64750 (0.0009) [2023-10-12 22:43:39,966][44959] Updated weights for policy 1, policy_version 64760 (0.0011) [2023-10-12 22:43:40,874][44958] Updated weights for policy 0, policy_version 64420 (0.0009) [2023-10-12 22:43:41,258][44958] Updated weights for policy 0, policy_version 64430 (0.0009) [2023-10-12 22:43:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132284416. Throughput: 0: 1648.9, 1: 1637.1. Samples: 33080596. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 22:43:41,443][43579] Avg episode reward: [(0, '277.490'), (1, '278.780')] [2023-10-12 22:43:41,630][44958] Updated weights for policy 0, policy_version 64440 (0.0010) [2023-10-12 22:43:44,172][44959] Updated weights for policy 1, policy_version 64770 (0.0008) [2023-10-12 22:43:44,535][44959] Updated weights for policy 1, policy_version 64780 (0.0008) [2023-10-12 22:43:44,900][44959] Updated weights for policy 1, policy_version 64790 (0.0008) [2023-10-12 22:43:45,265][44959] Updated weights for policy 1, policy_version 64800 (0.0008) [2023-10-12 22:43:45,658][44958] Updated weights for policy 0, policy_version 64450 (0.0010) [2023-10-12 22:43:46,042][44958] Updated weights for policy 0, policy_version 64460 (0.0011) [2023-10-12 22:43:46,404][44958] Updated weights for policy 0, policy_version 64470 (0.0010) [2023-10-12 22:43:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132349952. Throughput: 0: 1647.7, 1: 1650.5. Samples: 33100358. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 22:43:46,444][43579] Avg episode reward: [(0, '272.260'), (1, '280.950')] [2023-10-12 22:43:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000064800_66355200.pth... [2023-10-12 22:43:46,486][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000063264_64782336.pth [2023-10-12 22:43:46,771][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000064480_66027520.pth... [2023-10-12 22:43:46,773][44958] Updated weights for policy 0, policy_version 64480 (0.0009) [2023-10-12 22:43:46,810][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000062944_64454656.pth [2023-10-12 22:43:49,401][44959] Updated weights for policy 1, policy_version 64810 (0.0007) [2023-10-12 22:43:49,773][44959] Updated weights for policy 1, policy_version 64820 (0.0008) [2023-10-12 22:43:50,148][44959] Updated weights for policy 1, policy_version 64830 (0.0009) [2023-10-12 22:43:51,084][44958] Updated weights for policy 0, policy_version 64490 (0.0009) [2023-10-12 22:43:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132415488. Throughput: 0: 1646.3, 1: 1649.5. Samples: 33110620. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 22:43:51,444][43579] Avg episode reward: [(0, '269.130'), (1, '283.840')] [2023-10-12 22:43:51,454][44958] Updated weights for policy 0, policy_version 64500 (0.0008) [2023-10-12 22:43:51,830][44958] Updated weights for policy 0, policy_version 64510 (0.0009) [2023-10-12 22:43:54,561][44959] Updated weights for policy 1, policy_version 64840 (0.0009) [2023-10-12 22:43:54,929][44959] Updated weights for policy 1, policy_version 64850 (0.0007) [2023-10-12 22:43:55,293][44959] Updated weights for policy 1, policy_version 64860 (0.0008) [2023-10-12 22:43:56,019][44958] Updated weights for policy 0, policy_version 64520 (0.0007) [2023-10-12 22:43:56,394][44958] Updated weights for policy 0, policy_version 64530 (0.0010) [2023-10-12 22:43:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132481024. Throughput: 0: 1645.7, 1: 1643.1. Samples: 33130132. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 22:43:56,443][43579] Avg episode reward: [(0, '266.010'), (1, '280.450')] [2023-10-12 22:43:56,768][44958] Updated weights for policy 0, policy_version 64540 (0.0009) [2023-10-12 22:43:59,380][44959] Updated weights for policy 1, policy_version 64870 (0.0009) [2023-10-12 22:43:59,749][44959] Updated weights for policy 1, policy_version 64880 (0.0010) [2023-10-12 22:44:00,127][44959] Updated weights for policy 1, policy_version 64890 (0.0009) [2023-10-12 22:44:00,981][44958] Updated weights for policy 0, policy_version 64550 (0.0009) [2023-10-12 22:44:01,361][44958] Updated weights for policy 0, policy_version 64560 (0.0007) [2023-10-12 22:44:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132546560. Throughput: 0: 1644.4, 1: 1637.2. Samples: 33149316. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 22:44:01,444][43579] Avg episode reward: [(0, '268.830'), (1, '282.240')] [2023-10-12 22:44:01,732][44958] Updated weights for policy 0, policy_version 64570 (0.0009) [2023-10-12 22:44:04,353][44959] Updated weights for policy 1, policy_version 64900 (0.0007) [2023-10-12 22:44:04,726][44959] Updated weights for policy 1, policy_version 64910 (0.0007) [2023-10-12 22:44:05,096][44959] Updated weights for policy 1, policy_version 64920 (0.0007) [2023-10-12 22:44:05,718][44958] Updated weights for policy 0, policy_version 64580 (0.0010) [2023-10-12 22:44:06,083][44958] Updated weights for policy 0, policy_version 64590 (0.0008) [2023-10-12 22:44:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132612096. Throughput: 0: 1648.9, 1: 1637.4. Samples: 33159882. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 22:44:06,443][43579] Avg episode reward: [(0, '272.420'), (1, '284.070')] [2023-10-12 22:44:06,452][44958] Updated weights for policy 0, policy_version 64600 (0.0007) [2023-10-12 22:44:09,205][44959] Updated weights for policy 1, policy_version 64930 (0.0007) [2023-10-12 22:44:09,626][44959] Updated weights for policy 1, policy_version 64940 (0.0008) [2023-10-12 22:44:10,000][44959] Updated weights for policy 1, policy_version 64950 (0.0008) [2023-10-12 22:44:10,361][44959] Updated weights for policy 1, policy_version 64960 (0.0007) [2023-10-12 22:44:10,632][44958] Updated weights for policy 0, policy_version 64610 (0.0007) [2023-10-12 22:44:11,013][44958] Updated weights for policy 0, policy_version 64620 (0.0009) [2023-10-12 22:44:11,389][44958] Updated weights for policy 0, policy_version 64630 (0.0009) [2023-10-12 22:44:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132677632. Throughput: 0: 1648.6, 1: 1640.9. Samples: 33179268. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-12 22:44:11,443][43579] Avg episode reward: [(0, '273.710'), (1, '285.300')] [2023-10-12 22:44:11,757][44958] Updated weights for policy 0, policy_version 64640 (0.0009) [2023-10-12 22:44:14,521][44959] Updated weights for policy 1, policy_version 64970 (0.0010) [2023-10-12 22:44:14,885][44959] Updated weights for policy 1, policy_version 64980 (0.0009) [2023-10-12 22:44:15,252][44959] Updated weights for policy 1, policy_version 64990 (0.0009) [2023-10-12 22:44:16,040][44958] Updated weights for policy 0, policy_version 64650 (0.0007) [2023-10-12 22:44:16,411][44958] Updated weights for policy 0, policy_version 64660 (0.0007) [2023-10-12 22:44:16,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 132743168. Throughput: 0: 1644.0, 1: 1638.4. Samples: 33198822. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:44:16,444][43579] Avg episode reward: [(0, '279.460'), (1, '284.060')] [2023-10-12 22:44:16,791][44958] Updated weights for policy 0, policy_version 64670 (0.0009) [2023-10-12 22:44:19,458][44959] Updated weights for policy 1, policy_version 65000 (0.0008) [2023-10-12 22:44:19,830][44959] Updated weights for policy 1, policy_version 65010 (0.0009) [2023-10-12 22:44:20,192][44959] Updated weights for policy 1, policy_version 65020 (0.0008) [2023-10-12 22:44:20,860][44958] Updated weights for policy 0, policy_version 64680 (0.0009) [2023-10-12 22:44:21,233][44958] Updated weights for policy 0, policy_version 64690 (0.0009) [2023-10-12 22:44:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132808704. Throughput: 0: 1646.0, 1: 1643.9. Samples: 33209320. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:44:21,443][43579] Avg episode reward: [(0, '279.670'), (1, '282.160')] [2023-10-12 22:44:21,604][44958] Updated weights for policy 0, policy_version 64700 (0.0009) [2023-10-12 22:44:24,138][44959] Updated weights for policy 1, policy_version 65030 (0.0009) [2023-10-12 22:44:24,504][44959] Updated weights for policy 1, policy_version 65040 (0.0010) [2023-10-12 22:44:24,868][44959] Updated weights for policy 1, policy_version 65050 (0.0008) [2023-10-12 22:44:26,051][44958] Updated weights for policy 0, policy_version 64710 (0.0007) [2023-10-12 22:44:26,427][44958] Updated weights for policy 0, policy_version 64720 (0.0011) [2023-10-12 22:44:26,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132874240. Throughput: 0: 1635.2, 1: 1648.8. Samples: 33228380. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:44:26,443][43579] Avg episode reward: [(0, '280.260'), (1, '283.290')] [2023-10-12 22:44:26,789][44958] Updated weights for policy 0, policy_version 64730 (0.0010) [2023-10-12 22:44:28,959][44959] Updated weights for policy 1, policy_version 65060 (0.0007) [2023-10-12 22:44:29,325][44959] Updated weights for policy 1, policy_version 65070 (0.0007) [2023-10-12 22:44:29,692][44959] Updated weights for policy 1, policy_version 65080 (0.0009) [2023-10-12 22:44:30,999][44958] Updated weights for policy 0, policy_version 64740 (0.0010) [2023-10-12 22:44:31,363][44958] Updated weights for policy 0, policy_version 64750 (0.0007) [2023-10-12 22:44:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 132939776. Throughput: 0: 1636.6, 1: 1647.9. Samples: 33248162. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:44:31,444][43579] Avg episode reward: [(0, '277.760'), (1, '287.070')] [2023-10-12 22:44:31,742][44958] Updated weights for policy 0, policy_version 64760 (0.0007) [2023-10-12 22:44:33,857][44959] Updated weights for policy 1, policy_version 65090 (0.0009) [2023-10-12 22:44:34,228][44959] Updated weights for policy 1, policy_version 65100 (0.0011) [2023-10-12 22:44:34,595][44959] Updated weights for policy 1, policy_version 65110 (0.0008) [2023-10-12 22:44:34,957][44959] Updated weights for policy 1, policy_version 65120 (0.0008) [2023-10-12 22:44:35,714][44958] Updated weights for policy 0, policy_version 64770 (0.0010) [2023-10-12 22:44:36,089][44958] Updated weights for policy 0, policy_version 64780 (0.0007) [2023-10-12 22:44:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133005312. Throughput: 0: 1638.7, 1: 1643.8. Samples: 33258332. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:44:36,443][43579] Avg episode reward: [(0, '275.090'), (1, '286.000')] [2023-10-12 22:44:36,458][44958] Updated weights for policy 0, policy_version 64790 (0.0007) [2023-10-12 22:44:36,838][44958] Updated weights for policy 0, policy_version 64800 (0.0008) [2023-10-12 22:44:39,151][44959] Updated weights for policy 1, policy_version 65130 (0.0008) [2023-10-12 22:44:39,523][44959] Updated weights for policy 1, policy_version 65140 (0.0008) [2023-10-12 22:44:39,884][44959] Updated weights for policy 1, policy_version 65150 (0.0009) [2023-10-12 22:44:40,907][44958] Updated weights for policy 0, policy_version 64810 (0.0009) [2023-10-12 22:44:41,264][44958] Updated weights for policy 0, policy_version 64820 (0.0007) [2023-10-12 22:44:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133070848. Throughput: 0: 1635.5, 1: 1641.9. Samples: 33277612. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:44:41,444][43579] Avg episode reward: [(0, '273.210'), (1, '283.450')] [2023-10-12 22:44:41,637][44958] Updated weights for policy 0, policy_version 64830 (0.0007) [2023-10-12 22:44:44,106][44959] Updated weights for policy 1, policy_version 65160 (0.0008) [2023-10-12 22:44:44,483][44959] Updated weights for policy 1, policy_version 65170 (0.0009) [2023-10-12 22:44:44,845][44959] Updated weights for policy 1, policy_version 65180 (0.0008) [2023-10-12 22:44:45,774][44958] Updated weights for policy 0, policy_version 64840 (0.0009) [2023-10-12 22:44:46,149][44958] Updated weights for policy 0, policy_version 64850 (0.0008) [2023-10-12 22:44:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 133136384. Throughput: 0: 1632.6, 1: 1649.3. Samples: 33297000. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:44:46,443][43579] Avg episode reward: [(0, '273.040'), (1, '284.830')] [2023-10-12 22:44:46,509][44958] Updated weights for policy 0, policy_version 64860 (0.0010) [2023-10-12 22:44:49,092][44959] Updated weights for policy 1, policy_version 65190 (0.0011) [2023-10-12 22:44:49,455][44959] Updated weights for policy 1, policy_version 65200 (0.0008) [2023-10-12 22:44:49,828][44959] Updated weights for policy 1, policy_version 65210 (0.0007) [2023-10-12 22:44:50,804][44958] Updated weights for policy 0, policy_version 64870 (0.0008) [2023-10-12 22:44:51,169][44958] Updated weights for policy 0, policy_version 64880 (0.0009) [2023-10-12 22:44:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133201920. Throughput: 0: 1635.0, 1: 1642.5. Samples: 33307370. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:44:51,444][43579] Avg episode reward: [(0, '273.720'), (1, '287.790')] [2023-10-12 22:44:51,538][44958] Updated weights for policy 0, policy_version 64890 (0.0007) [2023-10-12 22:44:54,015][44959] Updated weights for policy 1, policy_version 65220 (0.0008) [2023-10-12 22:44:54,391][44959] Updated weights for policy 1, policy_version 65230 (0.0008) [2023-10-12 22:44:54,756][44959] Updated weights for policy 1, policy_version 65240 (0.0010) [2023-10-12 22:44:55,832][44958] Updated weights for policy 0, policy_version 64900 (0.0009) [2023-10-12 22:44:56,205][44958] Updated weights for policy 0, policy_version 64910 (0.0007) [2023-10-12 22:44:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133267456. Throughput: 0: 1634.2, 1: 1643.6. Samples: 33326766. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:44:56,443][43579] Avg episode reward: [(0, '276.510'), (1, '288.570')] [2023-10-12 22:44:56,590][44958] Updated weights for policy 0, policy_version 64920 (0.0009) [2023-10-12 22:44:59,056][44959] Updated weights for policy 1, policy_version 65250 (0.0009) [2023-10-12 22:44:59,461][44959] Updated weights for policy 1, policy_version 65260 (0.0007) [2023-10-12 22:44:59,832][44959] Updated weights for policy 1, policy_version 65270 (0.0010) [2023-10-12 22:45:00,190][44959] Updated weights for policy 1, policy_version 65280 (0.0008) [2023-10-12 22:45:00,645][44958] Updated weights for policy 0, policy_version 64930 (0.0011) [2023-10-12 22:45:01,022][44958] Updated weights for policy 0, policy_version 64940 (0.0010) [2023-10-12 22:45:01,400][44958] Updated weights for policy 0, policy_version 64950 (0.0011) [2023-10-12 22:45:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133332992. Throughput: 0: 1632.5, 1: 1644.2. Samples: 33346274. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:45:01,443][43579] Avg episode reward: [(0, '278.770'), (1, '289.680')] [2023-10-12 22:45:01,772][44958] Updated weights for policy 0, policy_version 64960 (0.0010) [2023-10-12 22:45:04,329][44959] Updated weights for policy 1, policy_version 65290 (0.0010) [2023-10-12 22:45:04,699][44959] Updated weights for policy 1, policy_version 65300 (0.0010) [2023-10-12 22:45:05,070][44959] Updated weights for policy 1, policy_version 65310 (0.0007) [2023-10-12 22:45:06,013][44958] Updated weights for policy 0, policy_version 64970 (0.0008) [2023-10-12 22:45:06,385][44958] Updated weights for policy 0, policy_version 64980 (0.0008) [2023-10-12 22:45:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133398528. Throughput: 0: 1638.8, 1: 1639.7. Samples: 33356850. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:45:06,443][43579] Avg episode reward: [(0, '278.430'), (1, '286.180')] [2023-10-12 22:45:06,753][44958] Updated weights for policy 0, policy_version 64990 (0.0009) [2023-10-12 22:45:09,283][44959] Updated weights for policy 1, policy_version 65320 (0.0008) [2023-10-12 22:45:09,663][44959] Updated weights for policy 1, policy_version 65330 (0.0010) [2023-10-12 22:45:10,023][44959] Updated weights for policy 1, policy_version 65340 (0.0010) [2023-10-12 22:45:11,084][44958] Updated weights for policy 0, policy_version 65000 (0.0009) [2023-10-12 22:45:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133464064. Throughput: 0: 1641.9, 1: 1642.0. Samples: 33376154. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:45:11,444][43579] Avg episode reward: [(0, '281.880'), (1, '287.240')] [2023-10-12 22:45:11,457][44958] Updated weights for policy 0, policy_version 65010 (0.0007) [2023-10-12 22:45:11,834][44958] Updated weights for policy 0, policy_version 65020 (0.0007) [2023-10-12 22:45:14,461][44959] Updated weights for policy 1, policy_version 65350 (0.0008) [2023-10-12 22:45:14,829][44959] Updated weights for policy 1, policy_version 65360 (0.0007) [2023-10-12 22:45:15,203][44959] Updated weights for policy 1, policy_version 65370 (0.0008) [2023-10-12 22:45:15,871][44958] Updated weights for policy 0, policy_version 65030 (0.0008) [2023-10-12 22:45:16,250][44958] Updated weights for policy 0, policy_version 65040 (0.0009) [2023-10-12 22:45:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 133529600. Throughput: 0: 1639.9, 1: 1634.4. Samples: 33395506. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:45:16,443][43579] Avg episode reward: [(0, '282.450'), (1, '286.030')] [2023-10-12 22:45:16,621][44958] Updated weights for policy 0, policy_version 65050 (0.0009) [2023-10-12 22:45:19,145][44959] Updated weights for policy 1, policy_version 65380 (0.0008) [2023-10-12 22:45:19,512][44959] Updated weights for policy 1, policy_version 65390 (0.0007) [2023-10-12 22:45:19,878][44959] Updated weights for policy 1, policy_version 65400 (0.0008) [2023-10-12 22:45:20,702][44958] Updated weights for policy 0, policy_version 65060 (0.0008) [2023-10-12 22:45:21,064][44958] Updated weights for policy 0, policy_version 65070 (0.0007) [2023-10-12 22:45:21,441][44958] Updated weights for policy 0, policy_version 65080 (0.0007) [2023-10-12 22:45:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133595136. Throughput: 0: 1645.6, 1: 1643.0. Samples: 33406320. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:45:21,443][43579] Avg episode reward: [(0, '283.380'), (1, '283.030')] [2023-10-12 22:45:24,138][44959] Updated weights for policy 1, policy_version 65410 (0.0009) [2023-10-12 22:45:24,501][44959] Updated weights for policy 1, policy_version 65420 (0.0008) [2023-10-12 22:45:24,868][44959] Updated weights for policy 1, policy_version 65430 (0.0009) [2023-10-12 22:45:25,230][44959] Updated weights for policy 1, policy_version 65440 (0.0007) [2023-10-12 22:45:25,720][44958] Updated weights for policy 0, policy_version 65090 (0.0009) [2023-10-12 22:45:26,127][44958] Updated weights for policy 0, policy_version 65100 (0.0009) [2023-10-12 22:45:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133660672. Throughput: 0: 1648.4, 1: 1641.2. Samples: 33425644. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:45:26,444][43579] Avg episode reward: [(0, '283.910'), (1, '279.460')] [2023-10-12 22:45:26,488][44958] Updated weights for policy 0, policy_version 65110 (0.0009) [2023-10-12 22:45:26,867][44958] Updated weights for policy 0, policy_version 65120 (0.0009) [2023-10-12 22:45:29,527][44959] Updated weights for policy 1, policy_version 65450 (0.0008) [2023-10-12 22:45:29,896][44959] Updated weights for policy 1, policy_version 65460 (0.0009) [2023-10-12 22:45:30,275][44959] Updated weights for policy 1, policy_version 65470 (0.0009) [2023-10-12 22:45:30,980][44958] Updated weights for policy 0, policy_version 65130 (0.0008) [2023-10-12 22:45:31,354][44958] Updated weights for policy 0, policy_version 65140 (0.0010) [2023-10-12 22:45:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133726208. Throughput: 0: 1649.0, 1: 1637.4. Samples: 33444888. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:45:31,443][43579] Avg episode reward: [(0, '283.300'), (1, '281.790')] [2023-10-12 22:45:31,721][44958] Updated weights for policy 0, policy_version 65150 (0.0010) [2023-10-12 22:45:34,302][44959] Updated weights for policy 1, policy_version 65480 (0.0008) [2023-10-12 22:45:34,674][44959] Updated weights for policy 1, policy_version 65490 (0.0008) [2023-10-12 22:45:35,040][44959] Updated weights for policy 1, policy_version 65500 (0.0008) [2023-10-12 22:45:35,879][44958] Updated weights for policy 0, policy_version 65160 (0.0008) [2023-10-12 22:45:36,252][44958] Updated weights for policy 0, policy_version 65170 (0.0009) [2023-10-12 22:45:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133791744. Throughput: 0: 1650.0, 1: 1644.7. Samples: 33455634. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:45:36,443][43579] Avg episode reward: [(0, '287.280'), (1, '277.910')] [2023-10-12 22:45:36,618][44958] Updated weights for policy 0, policy_version 65180 (0.0008) [2023-10-12 22:45:39,137][44959] Updated weights for policy 1, policy_version 65510 (0.0009) [2023-10-12 22:45:39,501][44959] Updated weights for policy 1, policy_version 65520 (0.0007) [2023-10-12 22:45:39,869][44959] Updated weights for policy 1, policy_version 65530 (0.0008) [2023-10-12 22:45:40,665][44958] Updated weights for policy 0, policy_version 65190 (0.0007) [2023-10-12 22:45:41,036][44958] Updated weights for policy 0, policy_version 65200 (0.0008) [2023-10-12 22:45:41,408][44958] Updated weights for policy 0, policy_version 65210 (0.0009) [2023-10-12 22:45:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133857280. Throughput: 0: 1646.3, 1: 1640.3. Samples: 33474664. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-12 22:45:41,444][43579] Avg episode reward: [(0, '287.670'), (1, '280.710')] [2023-10-12 22:45:44,207][44959] Updated weights for policy 1, policy_version 65540 (0.0007) [2023-10-12 22:45:44,581][44959] Updated weights for policy 1, policy_version 65550 (0.0008) [2023-10-12 22:45:44,951][44959] Updated weights for policy 1, policy_version 65560 (0.0008) [2023-10-12 22:45:45,736][44958] Updated weights for policy 0, policy_version 65220 (0.0008) [2023-10-12 22:45:46,102][44958] Updated weights for policy 0, policy_version 65230 (0.0008) [2023-10-12 22:45:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 133922816. Throughput: 0: 1644.1, 1: 1646.5. Samples: 33494352. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 22:45:46,443][43579] Avg episode reward: [(0, '288.230'), (1, '273.220')] [2023-10-12 22:45:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000065568_67141632.pth... [2023-10-12 22:45:46,474][44958] Updated weights for policy 0, policy_version 65240 (0.0009) [2023-10-12 22:45:46,481][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000064032_65568768.pth [2023-10-12 22:45:46,772][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000065248_66813952.pth... [2023-10-12 22:45:46,801][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000063712_65241088.pth [2023-10-12 22:45:48,876][44959] Updated weights for policy 1, policy_version 65570 (0.0008) [2023-10-12 22:45:49,244][44959] Updated weights for policy 1, policy_version 65580 (0.0009) [2023-10-12 22:45:49,606][44959] Updated weights for policy 1, policy_version 65590 (0.0007) [2023-10-12 22:45:49,962][44959] Updated weights for policy 1, policy_version 65600 (0.0007) [2023-10-12 22:45:50,451][44958] Updated weights for policy 0, policy_version 65250 (0.0008) [2023-10-12 22:45:50,823][44958] Updated weights for policy 0, policy_version 65260 (0.0008) [2023-10-12 22:45:51,208][44958] Updated weights for policy 0, policy_version 65270 (0.0010) [2023-10-12 22:45:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 133988352. Throughput: 0: 1639.6, 1: 1643.9. Samples: 33504610. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 22:45:51,444][43579] Avg episode reward: [(0, '287.930'), (1, '275.910')] [2023-10-12 22:45:51,584][44958] Updated weights for policy 0, policy_version 65280 (0.0009) [2023-10-12 22:45:53,794][44959] Updated weights for policy 1, policy_version 65610 (0.0010) [2023-10-12 22:45:54,159][44959] Updated weights for policy 1, policy_version 65620 (0.0007) [2023-10-12 22:45:54,519][44959] Updated weights for policy 1, policy_version 65630 (0.0008) [2023-10-12 22:45:55,937][44958] Updated weights for policy 0, policy_version 65290 (0.0008) [2023-10-12 22:45:56,303][44958] Updated weights for policy 0, policy_version 65300 (0.0008) [2023-10-12 22:45:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 134053888. Throughput: 0: 1636.9, 1: 1644.3. Samples: 33523808. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 22:45:56,443][43579] Avg episode reward: [(0, '287.460'), (1, '274.620')] [2023-10-12 22:45:56,677][44958] Updated weights for policy 0, policy_version 65310 (0.0007) [2023-10-12 22:45:58,807][44959] Updated weights for policy 1, policy_version 65640 (0.0009) [2023-10-12 22:45:59,177][44959] Updated weights for policy 1, policy_version 65650 (0.0009) [2023-10-12 22:45:59,555][44959] Updated weights for policy 1, policy_version 65660 (0.0007) [2023-10-12 22:46:00,750][44958] Updated weights for policy 0, policy_version 65320 (0.0007) [2023-10-12 22:46:01,119][44958] Updated weights for policy 0, policy_version 65330 (0.0009) [2023-10-12 22:46:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 134119424. Throughput: 0: 1636.7, 1: 1654.4. Samples: 33543606. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 22:46:01,444][43579] Avg episode reward: [(0, '282.080'), (1, '276.050')] [2023-10-12 22:46:01,501][44958] Updated weights for policy 0, policy_version 65340 (0.0008) [2023-10-12 22:46:03,583][44959] Updated weights for policy 1, policy_version 65670 (0.0009) [2023-10-12 22:46:03,949][44959] Updated weights for policy 1, policy_version 65680 (0.0008) [2023-10-12 22:46:04,322][44959] Updated weights for policy 1, policy_version 65690 (0.0008) [2023-10-12 22:46:05,796][44958] Updated weights for policy 0, policy_version 65350 (0.0009) [2023-10-12 22:46:06,168][44958] Updated weights for policy 0, policy_version 65360 (0.0009) [2023-10-12 22:46:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 134184960. Throughput: 0: 1635.0, 1: 1638.9. Samples: 33553646. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 22:46:06,443][43579] Avg episode reward: [(0, '279.610'), (1, '277.190')] [2023-10-12 22:46:06,543][44958] Updated weights for policy 0, policy_version 65370 (0.0008) [2023-10-12 22:46:08,535][44959] Updated weights for policy 1, policy_version 65700 (0.0008) [2023-10-12 22:46:08,903][44959] Updated weights for policy 1, policy_version 65710 (0.0009) [2023-10-12 22:46:09,269][44959] Updated weights for policy 1, policy_version 65720 (0.0010) [2023-10-12 22:46:10,804][44958] Updated weights for policy 0, policy_version 65380 (0.0009) [2023-10-12 22:46:11,199][44958] Updated weights for policy 0, policy_version 65390 (0.0009) [2023-10-12 22:46:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 134250496. Throughput: 0: 1635.7, 1: 1647.4. Samples: 33573382. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 22:46:11,443][43579] Avg episode reward: [(0, '271.600'), (1, '281.600')] [2023-10-12 22:46:11,581][44958] Updated weights for policy 0, policy_version 65400 (0.0008) [2023-10-12 22:46:13,533][44959] Updated weights for policy 1, policy_version 65730 (0.0009) [2023-10-12 22:46:13,895][44959] Updated weights for policy 1, policy_version 65740 (0.0008) [2023-10-12 22:46:14,262][44959] Updated weights for policy 1, policy_version 65750 (0.0008) [2023-10-12 22:46:14,630][44959] Updated weights for policy 1, policy_version 65760 (0.0008) [2023-10-12 22:46:15,677][44958] Updated weights for policy 0, policy_version 65410 (0.0009) [2023-10-12 22:46:16,045][44958] Updated weights for policy 0, policy_version 65420 (0.0009) [2023-10-12 22:46:16,411][44958] Updated weights for policy 0, policy_version 65430 (0.0010) [2023-10-12 22:46:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 134316032. Throughput: 0: 1638.8, 1: 1657.4. Samples: 33593220. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 22:46:16,444][43579] Avg episode reward: [(0, '267.960'), (1, '285.210')] [2023-10-12 22:46:16,789][44958] Updated weights for policy 0, policy_version 65440 (0.0007) [2023-10-12 22:46:18,951][44959] Updated weights for policy 1, policy_version 65770 (0.0009) [2023-10-12 22:46:19,314][44959] Updated weights for policy 1, policy_version 65780 (0.0010) [2023-10-12 22:46:19,677][44959] Updated weights for policy 1, policy_version 65790 (0.0010) [2023-10-12 22:46:21,103][44958] Updated weights for policy 0, policy_version 65450 (0.0009) [2023-10-12 22:46:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 134381568. Throughput: 0: 1636.0, 1: 1643.6. Samples: 33603218. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 22:46:21,443][43579] Avg episode reward: [(0, '266.870'), (1, '280.300')] [2023-10-12 22:46:21,472][44958] Updated weights for policy 0, policy_version 65460 (0.0010) [2023-10-12 22:46:21,847][44958] Updated weights for policy 0, policy_version 65470 (0.0010) [2023-10-12 22:46:24,017][44959] Updated weights for policy 1, policy_version 65800 (0.0011) [2023-10-12 22:46:24,391][44959] Updated weights for policy 1, policy_version 65810 (0.0007) [2023-10-12 22:46:24,764][44959] Updated weights for policy 1, policy_version 65820 (0.0007) [2023-10-12 22:46:25,844][44958] Updated weights for policy 0, policy_version 65480 (0.0008) [2023-10-12 22:46:26,218][44958] Updated weights for policy 0, policy_version 65490 (0.0007) [2023-10-12 22:46:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 134447104. Throughput: 0: 1640.5, 1: 1647.6. Samples: 33622630. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-12 22:46:26,444][43579] Avg episode reward: [(0, '267.610'), (1, '279.340')] [2023-10-12 22:46:26,594][44958] Updated weights for policy 0, policy_version 65500 (0.0011) [2023-10-12 22:46:29,012][44959] Updated weights for policy 1, policy_version 65830 (0.0009) [2023-10-12 22:46:29,400][44959] Updated weights for policy 1, policy_version 65840 (0.0007) [2023-10-12 22:46:29,769][44959] Updated weights for policy 1, policy_version 65850 (0.0007) [2023-10-12 22:46:30,928][44958] Updated weights for policy 0, policy_version 65510 (0.0008) [2023-10-12 22:46:31,301][44958] Updated weights for policy 0, policy_version 65520 (0.0007) [2023-10-12 22:46:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 134512640. Throughput: 0: 1644.3, 1: 1643.2. Samples: 33642290. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-12 22:46:31,443][43579] Avg episode reward: [(0, '272.060'), (1, '285.770')] [2023-10-12 22:46:31,677][44958] Updated weights for policy 0, policy_version 65530 (0.0008) [2023-10-12 22:46:33,825][44959] Updated weights for policy 1, policy_version 65860 (0.0008) [2023-10-12 22:46:34,203][44959] Updated weights for policy 1, policy_version 65870 (0.0008) [2023-10-12 22:46:34,568][44959] Updated weights for policy 1, policy_version 65880 (0.0009) [2023-10-12 22:46:35,839][44958] Updated weights for policy 0, policy_version 65540 (0.0011) [2023-10-12 22:46:36,220][44958] Updated weights for policy 0, policy_version 65550 (0.0010) [2023-10-12 22:46:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 134578176. Throughput: 0: 1643.5, 1: 1639.6. Samples: 33652348. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-12 22:46:36,444][43579] Avg episode reward: [(0, '271.360'), (1, '277.830')] [2023-10-12 22:46:36,603][44958] Updated weights for policy 0, policy_version 65560 (0.0008) [2023-10-12 22:46:38,799][44959] Updated weights for policy 1, policy_version 65890 (0.0008) [2023-10-12 22:46:39,173][44959] Updated weights for policy 1, policy_version 65900 (0.0007) [2023-10-12 22:46:39,547][44959] Updated weights for policy 1, policy_version 65910 (0.0009) [2023-10-12 22:46:39,903][44959] Updated weights for policy 1, policy_version 65920 (0.0008) [2023-10-12 22:46:40,751][44958] Updated weights for policy 0, policy_version 65570 (0.0009) [2023-10-12 22:46:41,131][44958] Updated weights for policy 0, policy_version 65580 (0.0008) [2023-10-12 22:46:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 134643712. Throughput: 0: 1651.1, 1: 1644.0. Samples: 33672084. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-12 22:46:41,444][43579] Avg episode reward: [(0, '274.500'), (1, '270.820')] [2023-10-12 22:46:41,506][44958] Updated weights for policy 0, policy_version 65590 (0.0010) [2023-10-12 22:46:41,887][44958] Updated weights for policy 0, policy_version 65600 (0.0010) [2023-10-12 22:46:44,025][44959] Updated weights for policy 1, policy_version 65930 (0.0008) [2023-10-12 22:46:44,384][44959] Updated weights for policy 1, policy_version 65940 (0.0007) [2023-10-12 22:46:44,744][44959] Updated weights for policy 1, policy_version 65950 (0.0007) [2023-10-12 22:46:46,043][44958] Updated weights for policy 0, policy_version 65610 (0.0007) [2023-10-12 22:46:46,415][44958] Updated weights for policy 0, policy_version 65620 (0.0009) [2023-10-12 22:46:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 134709248. Throughput: 0: 1648.5, 1: 1639.8. Samples: 33691580. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-12 22:46:46,443][43579] Avg episode reward: [(0, '278.980'), (1, '271.580')] [2023-10-12 22:46:46,778][44958] Updated weights for policy 0, policy_version 65630 (0.0007) [2023-10-12 22:46:48,865][44959] Updated weights for policy 1, policy_version 65960 (0.0010) [2023-10-12 22:46:49,234][44959] Updated weights for policy 1, policy_version 65970 (0.0007) [2023-10-12 22:46:49,605][44959] Updated weights for policy 1, policy_version 65980 (0.0008) [2023-10-12 22:46:50,939][44958] Updated weights for policy 0, policy_version 65640 (0.0008) [2023-10-12 22:46:51,306][44958] Updated weights for policy 0, policy_version 65650 (0.0008) [2023-10-12 22:46:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 134774784. Throughput: 0: 1644.1, 1: 1641.3. Samples: 33701490. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-12 22:46:51,444][43579] Avg episode reward: [(0, '279.890'), (1, '271.610')] [2023-10-12 22:46:51,679][44958] Updated weights for policy 0, policy_version 65660 (0.0010) [2023-10-12 22:46:53,935][44959] Updated weights for policy 1, policy_version 65990 (0.0010) [2023-10-12 22:46:54,293][44959] Updated weights for policy 1, policy_version 66000 (0.0009) [2023-10-12 22:46:54,664][44959] Updated weights for policy 1, policy_version 66010 (0.0007) [2023-10-12 22:46:55,917][44958] Updated weights for policy 0, policy_version 65670 (0.0010) [2023-10-12 22:46:56,311][44958] Updated weights for policy 0, policy_version 65680 (0.0010) [2023-10-12 22:46:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 134840320. Throughput: 0: 1638.8, 1: 1639.7. Samples: 33720916. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-12 22:46:56,444][43579] Avg episode reward: [(0, '281.650'), (1, '270.450')] [2023-10-12 22:46:56,674][44958] Updated weights for policy 0, policy_version 65690 (0.0009) [2023-10-12 22:46:58,867][44959] Updated weights for policy 1, policy_version 66020 (0.0007) [2023-10-12 22:46:59,228][44959] Updated weights for policy 1, policy_version 66030 (0.0008) [2023-10-12 22:46:59,604][44959] Updated weights for policy 1, policy_version 66040 (0.0007) [2023-10-12 22:47:00,834][44958] Updated weights for policy 0, policy_version 65700 (0.0008) [2023-10-12 22:47:01,206][44958] Updated weights for policy 0, policy_version 65710 (0.0008) [2023-10-12 22:47:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 134905856. Throughput: 0: 1632.4, 1: 1640.4. Samples: 33740492. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-12 22:47:01,443][43579] Avg episode reward: [(0, '280.500'), (1, '270.610')] [2023-10-12 22:47:01,574][44958] Updated weights for policy 0, policy_version 65720 (0.0010) [2023-10-12 22:47:03,589][44959] Updated weights for policy 1, policy_version 66050 (0.0008) [2023-10-12 22:47:03,953][44959] Updated weights for policy 1, policy_version 66060 (0.0008) [2023-10-12 22:47:04,324][44959] Updated weights for policy 1, policy_version 66070 (0.0007) [2023-10-12 22:47:04,684][44959] Updated weights for policy 1, policy_version 66080 (0.0008) [2023-10-12 22:47:05,849][44958] Updated weights for policy 0, policy_version 65730 (0.0008) [2023-10-12 22:47:06,223][44958] Updated weights for policy 0, policy_version 65740 (0.0009) [2023-10-12 22:47:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 134971392. Throughput: 0: 1634.6, 1: 1642.7. Samples: 33750698. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-12 22:47:06,443][43579] Avg episode reward: [(0, '280.380'), (1, '278.820')] [2023-10-12 22:47:06,590][44958] Updated weights for policy 0, policy_version 65750 (0.0009) [2023-10-12 22:47:06,955][44958] Updated weights for policy 0, policy_version 65760 (0.0009) [2023-10-12 22:47:08,831][44959] Updated weights for policy 1, policy_version 66090 (0.0011) [2023-10-12 22:47:09,190][44959] Updated weights for policy 1, policy_version 66100 (0.0008) [2023-10-12 22:47:09,556][44959] Updated weights for policy 1, policy_version 66110 (0.0009) [2023-10-12 22:47:11,036][44958] Updated weights for policy 0, policy_version 65770 (0.0010) [2023-10-12 22:47:11,413][44958] Updated weights for policy 0, policy_version 65780 (0.0007) [2023-10-12 22:47:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135036928. Throughput: 0: 1635.1, 1: 1648.3. Samples: 33770382. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-12 22:47:11,444][43579] Avg episode reward: [(0, '278.530'), (1, '281.180')] [2023-10-12 22:47:11,791][44958] Updated weights for policy 0, policy_version 65790 (0.0009) [2023-10-12 22:47:13,876][44959] Updated weights for policy 1, policy_version 66120 (0.0008) [2023-10-12 22:47:14,242][44959] Updated weights for policy 1, policy_version 66130 (0.0008) [2023-10-12 22:47:14,618][44959] Updated weights for policy 1, policy_version 66140 (0.0008) [2023-10-12 22:47:16,004][44958] Updated weights for policy 0, policy_version 65800 (0.0008) [2023-10-12 22:47:16,366][44958] Updated weights for policy 0, policy_version 65810 (0.0009) [2023-10-12 22:47:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135102464. Throughput: 0: 1631.1, 1: 1648.6. Samples: 33789876. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-12 22:47:16,444][43579] Avg episode reward: [(0, '276.020'), (1, '282.470')] [2023-10-12 22:47:16,737][44958] Updated weights for policy 0, policy_version 65820 (0.0010) [2023-10-12 22:47:18,720][44959] Updated weights for policy 1, policy_version 66150 (0.0009) [2023-10-12 22:47:19,089][44959] Updated weights for policy 1, policy_version 66160 (0.0007) [2023-10-12 22:47:19,459][44959] Updated weights for policy 1, policy_version 66170 (0.0009) [2023-10-12 22:47:21,015][44958] Updated weights for policy 0, policy_version 65830 (0.0008) [2023-10-12 22:47:21,394][44958] Updated weights for policy 0, policy_version 65840 (0.0009) [2023-10-12 22:47:21,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135168000. Throughput: 0: 1629.5, 1: 1646.9. Samples: 33799786. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-12 22:47:21,443][43579] Avg episode reward: [(0, '275.710'), (1, '283.940')] [2023-10-12 22:47:21,767][44958] Updated weights for policy 0, policy_version 65850 (0.0009) [2023-10-12 22:47:23,654][44959] Updated weights for policy 1, policy_version 66180 (0.0008) [2023-10-12 22:47:24,018][44959] Updated weights for policy 1, policy_version 66190 (0.0007) [2023-10-12 22:47:24,384][44959] Updated weights for policy 1, policy_version 66200 (0.0008) [2023-10-12 22:47:25,951][44958] Updated weights for policy 0, policy_version 65860 (0.0009) [2023-10-12 22:47:26,324][44958] Updated weights for policy 0, policy_version 65870 (0.0008) [2023-10-12 22:47:26,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135233536. Throughput: 0: 1629.7, 1: 1647.8. Samples: 33819568. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-12 22:47:26,443][43579] Avg episode reward: [(0, '276.440'), (1, '283.970')] [2023-10-12 22:47:26,699][44958] Updated weights for policy 0, policy_version 65880 (0.0008) [2023-10-12 22:47:28,341][44959] Updated weights for policy 1, policy_version 66210 (0.0007) [2023-10-12 22:47:28,709][44959] Updated weights for policy 1, policy_version 66220 (0.0008) [2023-10-12 22:47:29,086][44959] Updated weights for policy 1, policy_version 66230 (0.0008) [2023-10-12 22:47:29,452][44959] Updated weights for policy 1, policy_version 66240 (0.0008) [2023-10-12 22:47:30,976][44958] Updated weights for policy 0, policy_version 65890 (0.0009) [2023-10-12 22:47:31,341][44958] Updated weights for policy 0, policy_version 65900 (0.0009) [2023-10-12 22:47:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135299072. Throughput: 0: 1634.8, 1: 1651.3. Samples: 33839458. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-12 22:47:31,443][43579] Avg episode reward: [(0, '276.020'), (1, '282.800')] [2023-10-12 22:47:31,721][44958] Updated weights for policy 0, policy_version 65910 (0.0007) [2023-10-12 22:47:32,081][44958] Updated weights for policy 0, policy_version 65920 (0.0008) [2023-10-12 22:47:33,621][44959] Updated weights for policy 1, policy_version 66250 (0.0009) [2023-10-12 22:47:33,990][44959] Updated weights for policy 1, policy_version 66260 (0.0010) [2023-10-12 22:47:34,355][44959] Updated weights for policy 1, policy_version 66270 (0.0009) [2023-10-12 22:47:36,247][44958] Updated weights for policy 0, policy_version 65930 (0.0010) [2023-10-12 22:47:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135364608. Throughput: 0: 1629.6, 1: 1646.8. Samples: 33848926. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-12 22:47:36,443][43579] Avg episode reward: [(0, '275.940'), (1, '281.600')] [2023-10-12 22:47:36,606][44958] Updated weights for policy 0, policy_version 65940 (0.0008) [2023-10-12 22:47:36,988][44958] Updated weights for policy 0, policy_version 65950 (0.0008) [2023-10-12 22:47:38,530][44959] Updated weights for policy 1, policy_version 66280 (0.0007) [2023-10-12 22:47:38,904][44959] Updated weights for policy 1, policy_version 66290 (0.0009) [2023-10-12 22:47:39,272][44959] Updated weights for policy 1, policy_version 66300 (0.0010) [2023-10-12 22:47:41,079][44958] Updated weights for policy 0, policy_version 65960 (0.0011) [2023-10-12 22:47:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135430144. Throughput: 0: 1630.9, 1: 1649.6. Samples: 33868538. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-12 22:47:41,444][43579] Avg episode reward: [(0, '272.200'), (1, '279.540')] [2023-10-12 22:47:41,457][44958] Updated weights for policy 0, policy_version 65970 (0.0007) [2023-10-12 22:47:41,834][44958] Updated weights for policy 0, policy_version 65980 (0.0007) [2023-10-12 22:47:43,547][44959] Updated weights for policy 1, policy_version 66310 (0.0009) [2023-10-12 22:47:43,915][44959] Updated weights for policy 1, policy_version 66320 (0.0007) [2023-10-12 22:47:44,277][44959] Updated weights for policy 1, policy_version 66330 (0.0007) [2023-10-12 22:47:45,731][44958] Updated weights for policy 0, policy_version 65990 (0.0008) [2023-10-12 22:47:46,100][44958] Updated weights for policy 0, policy_version 66000 (0.0007) [2023-10-12 22:47:46,442][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135495680. Throughput: 0: 1641.8, 1: 1648.8. Samples: 33888568. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-12 22:47:46,443][43579] Avg episode reward: [(0, '271.410'), (1, '281.700')] [2023-10-12 22:47:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000066336_67928064.pth... [2023-10-12 22:47:46,469][44958] Updated weights for policy 0, policy_version 66010 (0.0007) [2023-10-12 22:47:46,483][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000064800_66355200.pth [2023-10-12 22:47:46,689][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000066016_67600384.pth... [2023-10-12 22:47:46,729][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000064480_66027520.pth [2023-10-12 22:47:48,361][44959] Updated weights for policy 1, policy_version 66340 (0.0009) [2023-10-12 22:47:48,722][44959] Updated weights for policy 1, policy_version 66350 (0.0009) [2023-10-12 22:47:49,098][44959] Updated weights for policy 1, policy_version 66360 (0.0007) [2023-10-12 22:47:50,535][44958] Updated weights for policy 0, policy_version 66020 (0.0008) [2023-10-12 22:47:50,897][44958] Updated weights for policy 0, policy_version 66030 (0.0009) [2023-10-12 22:47:51,271][44958] Updated weights for policy 0, policy_version 66040 (0.0007) [2023-10-12 22:47:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135561216. Throughput: 0: 1642.5, 1: 1639.2. Samples: 33898374. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-12 22:47:51,443][43579] Avg episode reward: [(0, '269.040'), (1, '281.460')] [2023-10-12 22:47:53,196][44959] Updated weights for policy 1, policy_version 66370 (0.0009) [2023-10-12 22:47:53,554][44959] Updated weights for policy 1, policy_version 66380 (0.0008) [2023-10-12 22:47:53,930][44959] Updated weights for policy 1, policy_version 66390 (0.0007) [2023-10-12 22:47:54,293][44959] Updated weights for policy 1, policy_version 66400 (0.0007) [2023-10-12 22:47:55,666][44958] Updated weights for policy 0, policy_version 66050 (0.0007) [2023-10-12 22:47:56,041][44958] Updated weights for policy 0, policy_version 66060 (0.0008) [2023-10-12 22:47:56,410][44958] Updated weights for policy 0, policy_version 66070 (0.0008) [2023-10-12 22:47:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135626752. Throughput: 0: 1642.5, 1: 1648.2. Samples: 33918462. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-12 22:47:56,443][43579] Avg episode reward: [(0, '269.230'), (1, '284.640')] [2023-10-12 22:47:56,787][44958] Updated weights for policy 0, policy_version 66080 (0.0009) [2023-10-12 22:47:58,600][44959] Updated weights for policy 1, policy_version 66410 (0.0007) [2023-10-12 22:47:58,979][44959] Updated weights for policy 1, policy_version 66420 (0.0010) [2023-10-12 22:47:59,351][44959] Updated weights for policy 1, policy_version 66430 (0.0008) [2023-10-12 22:48:00,736][44958] Updated weights for policy 0, policy_version 66090 (0.0008) [2023-10-12 22:48:01,107][44958] Updated weights for policy 0, policy_version 66100 (0.0007) [2023-10-12 22:48:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135692288. Throughput: 0: 1645.9, 1: 1651.4. Samples: 33938254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:01,443][43579] Avg episode reward: [(0, '272.860'), (1, '286.170')] [2023-10-12 22:48:01,482][44958] Updated weights for policy 0, policy_version 66110 (0.0007) [2023-10-12 22:48:03,445][44959] Updated weights for policy 1, policy_version 66440 (0.0010) [2023-10-12 22:48:03,812][44959] Updated weights for policy 1, policy_version 66450 (0.0009) [2023-10-12 22:48:04,172][44959] Updated weights for policy 1, policy_version 66460 (0.0011) [2023-10-12 22:48:05,617][44958] Updated weights for policy 0, policy_version 66120 (0.0008) [2023-10-12 22:48:05,999][44958] Updated weights for policy 0, policy_version 66130 (0.0007) [2023-10-12 22:48:06,361][44958] Updated weights for policy 0, policy_version 66140 (0.0008) [2023-10-12 22:48:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 135757824. Throughput: 0: 1656.3, 1: 1642.2. Samples: 33948216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:06,443][43579] Avg episode reward: [(0, '277.610'), (1, '287.690')] [2023-10-12 22:48:08,350][44959] Updated weights for policy 1, policy_version 66470 (0.0009) [2023-10-12 22:48:08,712][44959] Updated weights for policy 1, policy_version 66480 (0.0008) [2023-10-12 22:48:09,089][44959] Updated weights for policy 1, policy_version 66490 (0.0008) [2023-10-12 22:48:10,321][44958] Updated weights for policy 0, policy_version 66150 (0.0010) [2023-10-12 22:48:10,693][44958] Updated weights for policy 0, policy_version 66160 (0.0009) [2023-10-12 22:48:11,063][44958] Updated weights for policy 0, policy_version 66170 (0.0008) [2023-10-12 22:48:11,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 135856128. Throughput: 0: 1651.8, 1: 1647.6. Samples: 33968042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:11,444][43579] Avg episode reward: [(0, '281.810'), (1, '289.480')] [2023-10-12 22:48:13,338][44959] Updated weights for policy 1, policy_version 66500 (0.0007) [2023-10-12 22:48:13,711][44959] Updated weights for policy 1, policy_version 66510 (0.0008) [2023-10-12 22:48:14,083][44959] Updated weights for policy 1, policy_version 66520 (0.0008) [2023-10-12 22:48:15,254][44958] Updated weights for policy 0, policy_version 66180 (0.0007) [2023-10-12 22:48:15,621][44958] Updated weights for policy 0, policy_version 66190 (0.0008) [2023-10-12 22:48:15,992][44958] Updated weights for policy 0, policy_version 66200 (0.0008) [2023-10-12 22:48:16,442][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 135921664. Throughput: 0: 1643.6, 1: 1643.7. Samples: 33987386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:16,443][43579] Avg episode reward: [(0, '281.910'), (1, '288.350')] [2023-10-12 22:48:18,289][44959] Updated weights for policy 1, policy_version 66530 (0.0007) [2023-10-12 22:48:18,655][44959] Updated weights for policy 1, policy_version 66540 (0.0008) [2023-10-12 22:48:19,022][44959] Updated weights for policy 1, policy_version 66550 (0.0009) [2023-10-12 22:48:19,397][44959] Updated weights for policy 1, policy_version 66560 (0.0009) [2023-10-12 22:48:20,296][44958] Updated weights for policy 0, policy_version 66210 (0.0008) [2023-10-12 22:48:20,668][44958] Updated weights for policy 0, policy_version 66220 (0.0010) [2023-10-12 22:48:21,039][44958] Updated weights for policy 0, policy_version 66230 (0.0011) [2023-10-12 22:48:21,411][44958] Updated weights for policy 0, policy_version 66240 (0.0010) [2023-10-12 22:48:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 135987200. Throughput: 0: 1664.3, 1: 1645.0. Samples: 33997846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:21,443][43579] Avg episode reward: [(0, '284.550'), (1, '281.830')] [2023-10-12 22:48:23,355][44959] Updated weights for policy 1, policy_version 66570 (0.0008) [2023-10-12 22:48:23,723][44959] Updated weights for policy 1, policy_version 66580 (0.0009) [2023-10-12 22:48:24,086][44959] Updated weights for policy 1, policy_version 66590 (0.0010) [2023-10-12 22:48:25,732][44958] Updated weights for policy 0, policy_version 66250 (0.0010) [2023-10-12 22:48:26,116][44958] Updated weights for policy 0, policy_version 66260 (0.0011) [2023-10-12 22:48:26,443][43579] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136019968. Throughput: 0: 1660.5, 1: 1650.4. Samples: 34017528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:26,444][43579] Avg episode reward: [(0, '281.500'), (1, '281.620')] [2023-10-12 22:48:26,481][44958] Updated weights for policy 0, policy_version 66270 (0.0010) [2023-10-12 22:48:28,424][44959] Updated weights for policy 1, policy_version 66600 (0.0008) [2023-10-12 22:48:28,790][44959] Updated weights for policy 1, policy_version 66610 (0.0009) [2023-10-12 22:48:29,160][44959] Updated weights for policy 1, policy_version 66620 (0.0007) [2023-10-12 22:48:30,809][44958] Updated weights for policy 0, policy_version 66280 (0.0009) [2023-10-12 22:48:31,174][44958] Updated weights for policy 0, policy_version 66290 (0.0007) [2023-10-12 22:48:31,442][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136085504. Throughput: 0: 1646.6, 1: 1645.1. Samples: 34036696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:31,443][43579] Avg episode reward: [(0, '279.550'), (1, '281.840')] [2023-10-12 22:48:31,545][44958] Updated weights for policy 0, policy_version 66300 (0.0007) [2023-10-12 22:48:33,390][44959] Updated weights for policy 1, policy_version 66630 (0.0009) [2023-10-12 22:48:33,755][44959] Updated weights for policy 1, policy_version 66640 (0.0009) [2023-10-12 22:48:34,126][44959] Updated weights for policy 1, policy_version 66650 (0.0008) [2023-10-12 22:48:35,707][44958] Updated weights for policy 0, policy_version 66310 (0.0009) [2023-10-12 22:48:36,071][44958] Updated weights for policy 0, policy_version 66320 (0.0008) [2023-10-12 22:48:36,440][44958] Updated weights for policy 0, policy_version 66330 (0.0007) [2023-10-12 22:48:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136151040. Throughput: 0: 1651.1, 1: 1645.5. Samples: 34046720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:36,444][43579] Avg episode reward: [(0, '277.830'), (1, '280.730')] [2023-10-12 22:48:38,122][44959] Updated weights for policy 1, policy_version 66660 (0.0008) [2023-10-12 22:48:38,482][44959] Updated weights for policy 1, policy_version 66670 (0.0009) [2023-10-12 22:48:38,856][44959] Updated weights for policy 1, policy_version 66680 (0.0007) [2023-10-12 22:48:40,565][44958] Updated weights for policy 0, policy_version 66340 (0.0008) [2023-10-12 22:48:40,942][44958] Updated weights for policy 0, policy_version 66350 (0.0007) [2023-10-12 22:48:41,317][44958] Updated weights for policy 0, policy_version 66360 (0.0007) [2023-10-12 22:48:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 136216576. Throughput: 0: 1650.0, 1: 1646.0. Samples: 34066780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:41,443][43579] Avg episode reward: [(0, '274.730'), (1, '279.190')] [2023-10-12 22:48:42,457][44959] Updated weights for policy 1, policy_version 66690 (0.0009) [2023-10-12 22:48:42,859][44959] Updated weights for policy 1, policy_version 66700 (0.0009) [2023-10-12 22:48:43,235][44959] Updated weights for policy 1, policy_version 66710 (0.0008) [2023-10-12 22:48:43,609][44959] Updated weights for policy 1, policy_version 66720 (0.0008) [2023-10-12 22:48:44,977][44958] Updated weights for policy 0, policy_version 66370 (0.0009) [2023-10-12 22:48:45,348][44958] Updated weights for policy 0, policy_version 66380 (0.0007) [2023-10-12 22:48:45,726][44958] Updated weights for policy 0, policy_version 66390 (0.0009) [2023-10-12 22:48:46,095][44958] Updated weights for policy 0, policy_version 66400 (0.0008) [2023-10-12 22:48:46,442][43579] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 136314880. Throughput: 0: 1650.0, 1: 1670.5. Samples: 34087676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:46,443][43579] Avg episode reward: [(0, '278.440'), (1, '283.560')] [2023-10-12 22:48:47,945][44959] Updated weights for policy 1, policy_version 66730 (0.0011) [2023-10-12 22:48:48,309][44959] Updated weights for policy 1, policy_version 66740 (0.0009) [2023-10-12 22:48:48,681][44959] Updated weights for policy 1, policy_version 66750 (0.0007) [2023-10-12 22:48:50,131][44958] Updated weights for policy 0, policy_version 66410 (0.0011) [2023-10-12 22:48:50,505][44958] Updated weights for policy 0, policy_version 66420 (0.0008) [2023-10-12 22:48:50,880][44958] Updated weights for policy 0, policy_version 66430 (0.0009) [2023-10-12 22:48:51,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 136380416. Throughput: 0: 1667.1, 1: 1661.4. Samples: 34098000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:51,443][43579] Avg episode reward: [(0, '283.130'), (1, '281.330')] [2023-10-12 22:48:52,738][44959] Updated weights for policy 1, policy_version 66760 (0.0009) [2023-10-12 22:48:53,096][44959] Updated weights for policy 1, policy_version 66770 (0.0007) [2023-10-12 22:48:53,477][44959] Updated weights for policy 1, policy_version 66780 (0.0008) [2023-10-12 22:48:54,953][44958] Updated weights for policy 0, policy_version 66440 (0.0009) [2023-10-12 22:48:55,330][44958] Updated weights for policy 0, policy_version 66450 (0.0007) [2023-10-12 22:48:55,705][44958] Updated weights for policy 0, policy_version 66460 (0.0009) [2023-10-12 22:48:56,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 136445952. Throughput: 0: 1654.5, 1: 1673.0. Samples: 34117782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:48:56,443][43579] Avg episode reward: [(0, '280.040'), (1, '281.270')] [2023-10-12 22:48:57,626][44959] Updated weights for policy 1, policy_version 66790 (0.0010) [2023-10-12 22:48:57,992][44959] Updated weights for policy 1, policy_version 66800 (0.0007) [2023-10-12 22:48:58,361][44959] Updated weights for policy 1, policy_version 66810 (0.0007) [2023-10-12 22:49:00,010][44958] Updated weights for policy 0, policy_version 66470 (0.0008) [2023-10-12 22:49:00,383][44958] Updated weights for policy 0, policy_version 66480 (0.0010) [2023-10-12 22:49:00,755][44958] Updated weights for policy 0, policy_version 66490 (0.0009) [2023-10-12 22:49:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 136511488. Throughput: 0: 1657.4, 1: 1675.7. Samples: 34137378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:49:01,443][43579] Avg episode reward: [(0, '275.610'), (1, '274.340')] [2023-10-12 22:49:02,443][44959] Updated weights for policy 1, policy_version 66820 (0.0008) [2023-10-12 22:49:02,814][44959] Updated weights for policy 1, policy_version 66830 (0.0010) [2023-10-12 22:49:03,187][44959] Updated weights for policy 1, policy_version 66840 (0.0010) [2023-10-12 22:49:04,815][44958] Updated weights for policy 0, policy_version 66500 (0.0008) [2023-10-12 22:49:05,186][44958] Updated weights for policy 0, policy_version 66510 (0.0009) [2023-10-12 22:49:05,561][44958] Updated weights for policy 0, policy_version 66520 (0.0008) [2023-10-12 22:49:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 136577024. Throughput: 0: 1662.9, 1: 1663.4. Samples: 34147530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:49:06,444][43579] Avg episode reward: [(0, '275.780'), (1, '268.690')] [2023-10-12 22:49:07,227][44959] Updated weights for policy 1, policy_version 66850 (0.0010) [2023-10-12 22:49:07,603][44959] Updated weights for policy 1, policy_version 66860 (0.0008) [2023-10-12 22:49:07,970][44959] Updated weights for policy 1, policy_version 66870 (0.0008) [2023-10-12 22:49:08,341][44959] Updated weights for policy 1, policy_version 66880 (0.0008) [2023-10-12 22:49:09,526][44958] Updated weights for policy 0, policy_version 66530 (0.0008) [2023-10-12 22:49:09,903][44958] Updated weights for policy 0, policy_version 66540 (0.0009) [2023-10-12 22:49:10,273][44958] Updated weights for policy 0, policy_version 66550 (0.0009) [2023-10-12 22:49:10,644][44958] Updated weights for policy 0, policy_version 66560 (0.0008) [2023-10-12 22:49:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 136642560. Throughput: 0: 1656.9, 1: 1671.4. Samples: 34167302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:49:11,443][43579] Avg episode reward: [(0, '273.820'), (1, '269.180')] [2023-10-12 22:49:12,574][44959] Updated weights for policy 1, policy_version 66890 (0.0008) [2023-10-12 22:49:12,933][44959] Updated weights for policy 1, policy_version 66900 (0.0009) [2023-10-12 22:49:13,313][44959] Updated weights for policy 1, policy_version 66910 (0.0008) [2023-10-12 22:49:14,891][44958] Updated weights for policy 0, policy_version 66570 (0.0009) [2023-10-12 22:49:15,272][44958] Updated weights for policy 0, policy_version 66580 (0.0010) [2023-10-12 22:49:15,643][44958] Updated weights for policy 0, policy_version 66590 (0.0009) [2023-10-12 22:49:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 136708096. Throughput: 0: 1663.9, 1: 1680.0. Samples: 34187170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:49:16,443][43579] Avg episode reward: [(0, '273.850'), (1, '269.190')] [2023-10-12 22:49:17,506][44959] Updated weights for policy 1, policy_version 66920 (0.0010) [2023-10-12 22:49:17,875][44959] Updated weights for policy 1, policy_version 66930 (0.0010) [2023-10-12 22:49:18,241][44959] Updated weights for policy 1, policy_version 66940 (0.0010) [2023-10-12 22:49:19,624][44958] Updated weights for policy 0, policy_version 66600 (0.0010) [2023-10-12 22:49:19,990][44958] Updated weights for policy 0, policy_version 66610 (0.0008) [2023-10-12 22:49:20,365][44958] Updated weights for policy 0, policy_version 66620 (0.0010) [2023-10-12 22:49:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 136773632. Throughput: 0: 1678.4, 1: 1670.1. Samples: 34197406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:49:21,444][43579] Avg episode reward: [(0, '275.490'), (1, '273.650')] [2023-10-12 22:49:22,304][44959] Updated weights for policy 1, policy_version 66950 (0.0010) [2023-10-12 22:49:22,674][44959] Updated weights for policy 1, policy_version 66960 (0.0007) [2023-10-12 22:49:23,036][44959] Updated weights for policy 1, policy_version 66970 (0.0007) [2023-10-12 22:49:24,574][44958] Updated weights for policy 0, policy_version 66630 (0.0009) [2023-10-12 22:49:24,941][44958] Updated weights for policy 0, policy_version 66640 (0.0007) [2023-10-12 22:49:25,309][44958] Updated weights for policy 0, policy_version 66650 (0.0007) [2023-10-12 22:49:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 136839168. Throughput: 0: 1657.1, 1: 1676.9. Samples: 34216810. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-12 22:49:26,443][43579] Avg episode reward: [(0, '281.560'), (1, '280.690')] [2023-10-12 22:49:27,332][44959] Updated weights for policy 1, policy_version 66980 (0.0009) [2023-10-12 22:49:27,712][44959] Updated weights for policy 1, policy_version 66990 (0.0010) [2023-10-12 22:49:28,077][44959] Updated weights for policy 1, policy_version 67000 (0.0008) [2023-10-12 22:49:29,460][44958] Updated weights for policy 0, policy_version 66660 (0.0009) [2023-10-12 22:49:29,831][44958] Updated weights for policy 0, policy_version 66670 (0.0010) [2023-10-12 22:49:30,203][44958] Updated weights for policy 0, policy_version 66680 (0.0012) [2023-10-12 22:49:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 136904704. Throughput: 0: 1655.7, 1: 1648.8. Samples: 34236378. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-12 22:49:31,444][43579] Avg episode reward: [(0, '287.170'), (1, '281.450')] [2023-10-12 22:49:32,363][44959] Updated weights for policy 1, policy_version 67010 (0.0008) [2023-10-12 22:49:32,751][44959] Updated weights for policy 1, policy_version 67020 (0.0008) [2023-10-12 22:49:33,116][44959] Updated weights for policy 1, policy_version 67030 (0.0008) [2023-10-12 22:49:33,492][44959] Updated weights for policy 1, policy_version 67040 (0.0009) [2023-10-12 22:49:34,584][44958] Updated weights for policy 0, policy_version 66690 (0.0009) [2023-10-12 22:49:34,959][44958] Updated weights for policy 0, policy_version 66700 (0.0011) [2023-10-12 22:49:35,343][44958] Updated weights for policy 0, policy_version 66710 (0.0010) [2023-10-12 22:49:35,715][44958] Updated weights for policy 0, policy_version 66720 (0.0008) [2023-10-12 22:49:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 136970240. Throughput: 0: 1647.5, 1: 1649.5. Samples: 34246364. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-12 22:49:36,443][43579] Avg episode reward: [(0, '287.480'), (1, '282.010')] [2023-10-12 22:49:37,519][44959] Updated weights for policy 1, policy_version 67050 (0.0009) [2023-10-12 22:49:37,888][44959] Updated weights for policy 1, policy_version 67060 (0.0010) [2023-10-12 22:49:38,258][44959] Updated weights for policy 1, policy_version 67070 (0.0007) [2023-10-12 22:49:39,828][44958] Updated weights for policy 0, policy_version 66730 (0.0009) [2023-10-12 22:49:40,206][44958] Updated weights for policy 0, policy_version 66740 (0.0010) [2023-10-12 22:49:40,581][44958] Updated weights for policy 0, policy_version 66750 (0.0010) [2023-10-12 22:49:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 137035776. Throughput: 0: 1643.0, 1: 1647.7. Samples: 34265866. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-12 22:49:41,443][43579] Avg episode reward: [(0, '285.910'), (1, '279.150')] [2023-10-12 22:49:42,288][44959] Updated weights for policy 1, policy_version 67080 (0.0009) [2023-10-12 22:49:42,653][44959] Updated weights for policy 1, policy_version 67090 (0.0011) [2023-10-12 22:49:43,018][44959] Updated weights for policy 1, policy_version 67100 (0.0011) [2023-10-12 22:49:44,828][44958] Updated weights for policy 0, policy_version 66760 (0.0009) [2023-10-12 22:49:45,195][44958] Updated weights for policy 0, policy_version 66770 (0.0007) [2023-10-12 22:49:45,578][44958] Updated weights for policy 0, policy_version 66780 (0.0007) [2023-10-12 22:49:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137101312. Throughput: 0: 1644.0, 1: 1649.1. Samples: 34285568. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-12 22:49:46,443][43579] Avg episode reward: [(0, '283.900'), (1, '279.200')] [2023-10-12 22:49:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000067104_68714496.pth... [2023-10-12 22:49:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000066784_68386816.pth... [2023-10-12 22:49:46,483][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000065568_67141632.pth [2023-10-12 22:49:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000065248_66813952.pth [2023-10-12 22:49:47,225][44959] Updated weights for policy 1, policy_version 67110 (0.0009) [2023-10-12 22:49:47,595][44959] Updated weights for policy 1, policy_version 67120 (0.0007) [2023-10-12 22:49:47,968][44959] Updated weights for policy 1, policy_version 67130 (0.0008) [2023-10-12 22:49:49,834][44958] Updated weights for policy 0, policy_version 66790 (0.0008) [2023-10-12 22:49:50,210][44958] Updated weights for policy 0, policy_version 66800 (0.0008) [2023-10-12 22:49:50,592][44958] Updated weights for policy 0, policy_version 66810 (0.0007) [2023-10-12 22:49:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137166848. Throughput: 0: 1643.1, 1: 1647.9. Samples: 34295626. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-12 22:49:51,443][43579] Avg episode reward: [(0, '283.840'), (1, '277.610')] [2023-10-12 22:49:52,023][44959] Updated weights for policy 1, policy_version 67140 (0.0009) [2023-10-12 22:49:52,402][44959] Updated weights for policy 1, policy_version 67150 (0.0008) [2023-10-12 22:49:52,769][44959] Updated weights for policy 1, policy_version 67160 (0.0009) [2023-10-12 22:49:54,629][44958] Updated weights for policy 0, policy_version 66820 (0.0009) [2023-10-12 22:49:54,998][44958] Updated weights for policy 0, policy_version 66830 (0.0009) [2023-10-12 22:49:55,378][44958] Updated weights for policy 0, policy_version 66840 (0.0008) [2023-10-12 22:49:56,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137232384. Throughput: 0: 1636.0, 1: 1652.4. Samples: 34315276. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-12 22:49:56,443][43579] Avg episode reward: [(0, '280.020'), (1, '277.370')] [2023-10-12 22:49:56,832][44959] Updated weights for policy 1, policy_version 67170 (0.0011) [2023-10-12 22:49:57,202][44959] Updated weights for policy 1, policy_version 67180 (0.0007) [2023-10-12 22:49:57,567][44959] Updated weights for policy 1, policy_version 67190 (0.0007) [2023-10-12 22:49:57,938][44959] Updated weights for policy 1, policy_version 67200 (0.0010) [2023-10-12 22:49:59,599][44958] Updated weights for policy 0, policy_version 66850 (0.0007) [2023-10-12 22:49:59,980][44958] Updated weights for policy 0, policy_version 66860 (0.0007) [2023-10-12 22:50:00,349][44958] Updated weights for policy 0, policy_version 66870 (0.0009) [2023-10-12 22:50:00,724][44958] Updated weights for policy 0, policy_version 66880 (0.0007) [2023-10-12 22:50:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137297920. Throughput: 0: 1634.0, 1: 1648.9. Samples: 34334900. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-12 22:50:01,443][43579] Avg episode reward: [(0, '281.740'), (1, '281.260')] [2023-10-12 22:50:02,262][44959] Updated weights for policy 1, policy_version 67210 (0.0007) [2023-10-12 22:50:02,636][44959] Updated weights for policy 1, policy_version 67220 (0.0008) [2023-10-12 22:50:03,001][44959] Updated weights for policy 1, policy_version 67230 (0.0009) [2023-10-12 22:50:04,939][44958] Updated weights for policy 0, policy_version 66890 (0.0008) [2023-10-12 22:50:05,322][44958] Updated weights for policy 0, policy_version 66900 (0.0008) [2023-10-12 22:50:05,700][44958] Updated weights for policy 0, policy_version 66910 (0.0010) [2023-10-12 22:50:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137363456. Throughput: 0: 1630.6, 1: 1650.5. Samples: 34345056. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-12 22:50:06,444][43579] Avg episode reward: [(0, '281.290'), (1, '286.580')] [2023-10-12 22:50:07,004][44959] Updated weights for policy 1, policy_version 67240 (0.0008) [2023-10-12 22:50:07,381][44959] Updated weights for policy 1, policy_version 67250 (0.0008) [2023-10-12 22:50:07,748][44959] Updated weights for policy 1, policy_version 67260 (0.0008) [2023-10-12 22:50:09,727][44958] Updated weights for policy 0, policy_version 66920 (0.0009) [2023-10-12 22:50:10,097][44958] Updated weights for policy 0, policy_version 66930 (0.0008) [2023-10-12 22:50:10,469][44958] Updated weights for policy 0, policy_version 66940 (0.0007) [2023-10-12 22:50:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137428992. Throughput: 0: 1636.4, 1: 1650.4. Samples: 34364714. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-12 22:50:11,443][43579] Avg episode reward: [(0, '280.530'), (1, '286.870')] [2023-10-12 22:50:11,901][44959] Updated weights for policy 1, policy_version 67270 (0.0010) [2023-10-12 22:50:12,271][44959] Updated weights for policy 1, policy_version 67280 (0.0009) [2023-10-12 22:50:12,641][44959] Updated weights for policy 1, policy_version 67290 (0.0007) [2023-10-12 22:50:14,603][44958] Updated weights for policy 0, policy_version 66950 (0.0009) [2023-10-12 22:50:14,972][44958] Updated weights for policy 0, policy_version 66960 (0.0007) [2023-10-12 22:50:15,364][44958] Updated weights for policy 0, policy_version 66970 (0.0008) [2023-10-12 22:50:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137494528. Throughput: 0: 1638.5, 1: 1653.1. Samples: 34384498. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-12 22:50:16,444][43579] Avg episode reward: [(0, '282.200'), (1, '284.660')] [2023-10-12 22:50:17,021][44959] Updated weights for policy 1, policy_version 67300 (0.0007) [2023-10-12 22:50:17,426][44959] Updated weights for policy 1, policy_version 67310 (0.0008) [2023-10-12 22:50:17,790][44959] Updated weights for policy 1, policy_version 67320 (0.0007) [2023-10-12 22:50:19,451][44958] Updated weights for policy 0, policy_version 66980 (0.0011) [2023-10-12 22:50:19,823][44958] Updated weights for policy 0, policy_version 66990 (0.0007) [2023-10-12 22:50:20,188][44958] Updated weights for policy 0, policy_version 67000 (0.0009) [2023-10-12 22:50:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137560064. Throughput: 0: 1643.1, 1: 1650.2. Samples: 34394562. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-12 22:50:21,443][43579] Avg episode reward: [(0, '276.550'), (1, '286.470')] [2023-10-12 22:50:21,782][44959] Updated weights for policy 1, policy_version 67330 (0.0009) [2023-10-12 22:50:22,159][44959] Updated weights for policy 1, policy_version 67340 (0.0009) [2023-10-12 22:50:22,531][44959] Updated weights for policy 1, policy_version 67350 (0.0009) [2023-10-12 22:50:22,898][44959] Updated weights for policy 1, policy_version 67360 (0.0007) [2023-10-12 22:50:24,467][44958] Updated weights for policy 0, policy_version 67010 (0.0010) [2023-10-12 22:50:24,843][44958] Updated weights for policy 0, policy_version 67020 (0.0009) [2023-10-12 22:50:25,211][44958] Updated weights for policy 0, policy_version 67030 (0.0010) [2023-10-12 22:50:25,585][44958] Updated weights for policy 0, policy_version 67040 (0.0008) [2023-10-12 22:50:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137625600. Throughput: 0: 1641.7, 1: 1652.4. Samples: 34414104. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-12 22:50:26,443][43579] Avg episode reward: [(0, '271.220'), (1, '285.750')] [2023-10-12 22:50:26,976][44959] Updated weights for policy 1, policy_version 67370 (0.0010) [2023-10-12 22:50:27,338][44959] Updated weights for policy 1, policy_version 67380 (0.0010) [2023-10-12 22:50:27,720][44959] Updated weights for policy 1, policy_version 67390 (0.0008) [2023-10-12 22:50:29,744][44958] Updated weights for policy 0, policy_version 67050 (0.0008) [2023-10-12 22:50:30,120][44958] Updated weights for policy 0, policy_version 67060 (0.0007) [2023-10-12 22:50:30,496][44958] Updated weights for policy 0, policy_version 67070 (0.0007) [2023-10-12 22:50:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137691136. Throughput: 0: 1645.0, 1: 1652.5. Samples: 34433954. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-12 22:50:31,444][43579] Avg episode reward: [(0, '273.930'), (1, '277.680')] [2023-10-12 22:50:31,867][44959] Updated weights for policy 1, policy_version 67400 (0.0010) [2023-10-12 22:50:32,235][44959] Updated weights for policy 1, policy_version 67410 (0.0007) [2023-10-12 22:50:32,599][44959] Updated weights for policy 1, policy_version 67420 (0.0010) [2023-10-12 22:50:34,739][44958] Updated weights for policy 0, policy_version 67080 (0.0007) [2023-10-12 22:50:35,114][44958] Updated weights for policy 0, policy_version 67090 (0.0009) [2023-10-12 22:50:35,485][44958] Updated weights for policy 0, policy_version 67100 (0.0007) [2023-10-12 22:50:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137756672. Throughput: 0: 1645.5, 1: 1653.0. Samples: 34444056. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-12 22:50:36,444][43579] Avg episode reward: [(0, '276.370'), (1, '276.470')] [2023-10-12 22:50:36,758][44959] Updated weights for policy 1, policy_version 67430 (0.0010) [2023-10-12 22:50:37,123][44959] Updated weights for policy 1, policy_version 67440 (0.0010) [2023-10-12 22:50:37,504][44959] Updated weights for policy 1, policy_version 67450 (0.0009) [2023-10-12 22:50:39,605][44958] Updated weights for policy 0, policy_version 67110 (0.0010) [2023-10-12 22:50:39,978][44958] Updated weights for policy 0, policy_version 67120 (0.0009) [2023-10-12 22:50:40,348][44958] Updated weights for policy 0, policy_version 67130 (0.0007) [2023-10-12 22:50:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137822208. Throughput: 0: 1642.8, 1: 1650.4. Samples: 34463470. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-12 22:50:41,444][43579] Avg episode reward: [(0, '276.950'), (1, '274.890')] [2023-10-12 22:50:41,483][44959] Updated weights for policy 1, policy_version 67460 (0.0008) [2023-10-12 22:50:41,858][44959] Updated weights for policy 1, policy_version 67470 (0.0007) [2023-10-12 22:50:42,225][44959] Updated weights for policy 1, policy_version 67480 (0.0008) [2023-10-12 22:50:44,699][44958] Updated weights for policy 0, policy_version 67140 (0.0008) [2023-10-12 22:50:45,086][44958] Updated weights for policy 0, policy_version 67150 (0.0009) [2023-10-12 22:50:45,452][44958] Updated weights for policy 0, policy_version 67160 (0.0007) [2023-10-12 22:50:46,431][44959] Updated weights for policy 1, policy_version 67490 (0.0008) [2023-10-12 22:50:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137887744. Throughput: 0: 1643.2, 1: 1650.9. Samples: 34483134. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-12 22:50:46,444][43579] Avg episode reward: [(0, '278.140'), (1, '277.190')] [2023-10-12 22:50:46,797][44959] Updated weights for policy 1, policy_version 67500 (0.0009) [2023-10-12 22:50:47,155][44959] Updated weights for policy 1, policy_version 67510 (0.0008) [2023-10-12 22:50:47,523][44959] Updated weights for policy 1, policy_version 67520 (0.0008) [2023-10-12 22:50:49,488][44958] Updated weights for policy 0, policy_version 67170 (0.0010) [2023-10-12 22:50:49,861][44958] Updated weights for policy 0, policy_version 67180 (0.0007) [2023-10-12 22:50:50,231][44958] Updated weights for policy 0, policy_version 67190 (0.0008) [2023-10-12 22:50:50,601][44958] Updated weights for policy 0, policy_version 67200 (0.0008) [2023-10-12 22:50:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 137953280. Throughput: 0: 1647.0, 1: 1651.1. Samples: 34493470. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-12 22:50:51,443][43579] Avg episode reward: [(0, '282.560'), (1, '271.640')] [2023-10-12 22:50:51,917][44959] Updated weights for policy 1, policy_version 67530 (0.0010) [2023-10-12 22:50:52,288][44959] Updated weights for policy 1, policy_version 67540 (0.0008) [2023-10-12 22:50:52,645][44959] Updated weights for policy 1, policy_version 67550 (0.0008) [2023-10-12 22:50:54,609][44958] Updated weights for policy 0, policy_version 67210 (0.0010) [2023-10-12 22:50:54,980][44958] Updated weights for policy 0, policy_version 67220 (0.0008) [2023-10-12 22:50:55,357][44958] Updated weights for policy 0, policy_version 67230 (0.0008) [2023-10-12 22:50:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138018816. Throughput: 0: 1644.5, 1: 1646.3. Samples: 34512800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:50:56,444][43579] Avg episode reward: [(0, '284.050'), (1, '273.690')] [2023-10-12 22:50:56,647][44959] Updated weights for policy 1, policy_version 67560 (0.0009) [2023-10-12 22:50:57,005][44959] Updated weights for policy 1, policy_version 67570 (0.0009) [2023-10-12 22:50:57,378][44959] Updated weights for policy 1, policy_version 67580 (0.0008) [2023-10-12 22:50:59,645][44958] Updated weights for policy 0, policy_version 67240 (0.0008) [2023-10-12 22:51:00,013][44958] Updated weights for policy 0, policy_version 67250 (0.0009) [2023-10-12 22:51:00,384][44958] Updated weights for policy 0, policy_version 67260 (0.0009) [2023-10-12 22:51:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138084352. Throughput: 0: 1651.1, 1: 1646.3. Samples: 34532878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:51:01,443][43579] Avg episode reward: [(0, '282.150'), (1, '274.690')] [2023-10-12 22:51:01,670][44959] Updated weights for policy 1, policy_version 67590 (0.0009) [2023-10-12 22:51:02,037][44959] Updated weights for policy 1, policy_version 67600 (0.0009) [2023-10-12 22:51:02,410][44959] Updated weights for policy 1, policy_version 67610 (0.0009) [2023-10-12 22:51:04,363][44958] Updated weights for policy 0, policy_version 67270 (0.0009) [2023-10-12 22:51:04,743][44958] Updated weights for policy 0, policy_version 67280 (0.0010) [2023-10-12 22:51:05,126][44958] Updated weights for policy 0, policy_version 67290 (0.0009) [2023-10-12 22:51:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138149888. Throughput: 0: 1646.6, 1: 1652.5. Samples: 34543022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:51:06,443][43579] Avg episode reward: [(0, '281.930'), (1, '273.320')] [2023-10-12 22:51:06,664][44959] Updated weights for policy 1, policy_version 67620 (0.0009) [2023-10-12 22:51:07,059][44959] Updated weights for policy 1, policy_version 67630 (0.0009) [2023-10-12 22:51:07,423][44959] Updated weights for policy 1, policy_version 67640 (0.0007) [2023-10-12 22:51:09,441][44958] Updated weights for policy 0, policy_version 67300 (0.0007) [2023-10-12 22:51:09,814][44958] Updated weights for policy 0, policy_version 67310 (0.0008) [2023-10-12 22:51:10,189][44958] Updated weights for policy 0, policy_version 67320 (0.0007) [2023-10-12 22:51:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138215424. Throughput: 0: 1643.8, 1: 1645.6. Samples: 34562128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:51:11,444][43579] Avg episode reward: [(0, '279.420'), (1, '273.650')] [2023-10-12 22:51:11,611][44959] Updated weights for policy 1, policy_version 67650 (0.0007) [2023-10-12 22:51:11,981][44959] Updated weights for policy 1, policy_version 67660 (0.0009) [2023-10-12 22:51:12,345][44959] Updated weights for policy 1, policy_version 67670 (0.0008) [2023-10-12 22:51:12,704][44959] Updated weights for policy 1, policy_version 67680 (0.0007) [2023-10-12 22:51:14,587][44958] Updated weights for policy 0, policy_version 67330 (0.0009) [2023-10-12 22:51:14,956][44958] Updated weights for policy 0, policy_version 67340 (0.0011) [2023-10-12 22:51:15,325][44958] Updated weights for policy 0, policy_version 67350 (0.0008) [2023-10-12 22:51:15,695][44958] Updated weights for policy 0, policy_version 67360 (0.0008) [2023-10-12 22:51:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138280960. Throughput: 0: 1642.0, 1: 1646.0. Samples: 34581912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:51:16,443][43579] Avg episode reward: [(0, '284.340'), (1, '273.710')] [2023-10-12 22:51:16,904][44959] Updated weights for policy 1, policy_version 67690 (0.0008) [2023-10-12 22:51:17,283][44959] Updated weights for policy 1, policy_version 67700 (0.0009) [2023-10-12 22:51:17,643][44959] Updated weights for policy 1, policy_version 67710 (0.0008) [2023-10-12 22:51:19,912][44958] Updated weights for policy 0, policy_version 67370 (0.0010) [2023-10-12 22:51:20,290][44958] Updated weights for policy 0, policy_version 67380 (0.0008) [2023-10-12 22:51:20,653][44958] Updated weights for policy 0, policy_version 67390 (0.0008) [2023-10-12 22:51:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138346496. Throughput: 0: 1637.9, 1: 1647.4. Samples: 34591892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:51:21,443][43579] Avg episode reward: [(0, '281.090'), (1, '280.190')] [2023-10-12 22:51:22,051][44959] Updated weights for policy 1, policy_version 67720 (0.0007) [2023-10-12 22:51:22,423][44959] Updated weights for policy 1, policy_version 67730 (0.0007) [2023-10-12 22:51:22,790][44959] Updated weights for policy 1, policy_version 67740 (0.0008) [2023-10-12 22:51:24,912][44958] Updated weights for policy 0, policy_version 67400 (0.0007) [2023-10-12 22:51:25,286][44958] Updated weights for policy 0, policy_version 67410 (0.0007) [2023-10-12 22:51:25,661][44958] Updated weights for policy 0, policy_version 67420 (0.0008) [2023-10-12 22:51:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 138412032. Throughput: 0: 1642.4, 1: 1642.4. Samples: 34611288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:51:26,444][43579] Avg episode reward: [(0, '283.810'), (1, '280.240')] [2023-10-12 22:51:27,009][44959] Updated weights for policy 1, policy_version 67750 (0.0009) [2023-10-12 22:51:27,374][44959] Updated weights for policy 1, policy_version 67760 (0.0007) [2023-10-12 22:51:27,733][44959] Updated weights for policy 1, policy_version 67770 (0.0007) [2023-10-12 22:51:29,796][44958] Updated weights for policy 0, policy_version 67430 (0.0008) [2023-10-12 22:51:30,176][44958] Updated weights for policy 0, policy_version 67440 (0.0007) [2023-10-12 22:51:30,551][44958] Updated weights for policy 0, policy_version 67450 (0.0010) [2023-10-12 22:51:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138477568. Throughput: 0: 1638.9, 1: 1645.3. Samples: 34630920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:51:31,443][43579] Avg episode reward: [(0, '285.040'), (1, '278.400')] [2023-10-12 22:51:31,716][44959] Updated weights for policy 1, policy_version 67780 (0.0010) [2023-10-12 22:51:32,081][44959] Updated weights for policy 1, policy_version 67790 (0.0007) [2023-10-12 22:51:32,451][44959] Updated weights for policy 1, policy_version 67800 (0.0008) [2023-10-12 22:51:34,528][44958] Updated weights for policy 0, policy_version 67460 (0.0010) [2023-10-12 22:51:34,912][44958] Updated weights for policy 0, policy_version 67470 (0.0007) [2023-10-12 22:51:35,279][44958] Updated weights for policy 0, policy_version 67480 (0.0009) [2023-10-12 22:51:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138543104. Throughput: 0: 1633.1, 1: 1646.0. Samples: 34641032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:51:36,443][43579] Avg episode reward: [(0, '283.250'), (1, '274.400')] [2023-10-12 22:51:36,734][44959] Updated weights for policy 1, policy_version 67810 (0.0009) [2023-10-12 22:51:37,096][44959] Updated weights for policy 1, policy_version 67820 (0.0007) [2023-10-12 22:51:37,461][44959] Updated weights for policy 1, policy_version 67830 (0.0010) [2023-10-12 22:51:37,825][44959] Updated weights for policy 1, policy_version 67840 (0.0007) [2023-10-12 22:51:39,545][44958] Updated weights for policy 0, policy_version 67490 (0.0008) [2023-10-12 22:51:39,926][44958] Updated weights for policy 0, policy_version 67500 (0.0008) [2023-10-12 22:51:40,295][44958] Updated weights for policy 0, policy_version 67510 (0.0008) [2023-10-12 22:51:40,657][44958] Updated weights for policy 0, policy_version 67520 (0.0008) [2023-10-12 22:51:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138608640. Throughput: 0: 1635.5, 1: 1648.0. Samples: 34660560. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-12 22:51:41,444][43579] Avg episode reward: [(0, '281.490'), (1, '276.200')] [2023-10-12 22:51:41,863][44959] Updated weights for policy 1, policy_version 67850 (0.0009) [2023-10-12 22:51:42,245][44959] Updated weights for policy 1, policy_version 67860 (0.0009) [2023-10-12 22:51:42,607][44959] Updated weights for policy 1, policy_version 67870 (0.0007) [2023-10-12 22:51:44,564][44958] Updated weights for policy 0, policy_version 67530 (0.0008) [2023-10-12 22:51:44,933][44958] Updated weights for policy 0, policy_version 67540 (0.0009) [2023-10-12 22:51:45,295][44958] Updated weights for policy 0, policy_version 67550 (0.0008) [2023-10-12 22:51:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138674176. Throughput: 0: 1630.7, 1: 1650.6. Samples: 34680538. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-12 22:51:46,444][43579] Avg episode reward: [(0, '279.930'), (1, '274.130')] [2023-10-12 22:51:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000067552_69173248.pth... [2023-10-12 22:51:46,488][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000066016_67600384.pth [2023-10-12 22:51:46,674][44959] Updated weights for policy 1, policy_version 67880 (0.0009) [2023-10-12 22:51:47,042][44959] Updated weights for policy 1, policy_version 67890 (0.0009) [2023-10-12 22:51:47,414][44959] Updated weights for policy 1, policy_version 67900 (0.0008) [2023-10-12 22:51:47,553][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000067904_69533696.pth... [2023-10-12 22:51:47,583][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000066336_67928064.pth [2023-10-12 22:51:49,490][44958] Updated weights for policy 0, policy_version 67560 (0.0007) [2023-10-12 22:51:49,858][44958] Updated weights for policy 0, policy_version 67570 (0.0008) [2023-10-12 22:51:50,229][44958] Updated weights for policy 0, policy_version 67580 (0.0008) [2023-10-12 22:51:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138739712. Throughput: 0: 1635.1, 1: 1647.8. Samples: 34690750. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-12 22:51:51,443][43579] Avg episode reward: [(0, '281.180'), (1, '273.350')] [2023-10-12 22:51:51,622][44959] Updated weights for policy 1, policy_version 67910 (0.0007) [2023-10-12 22:51:51,988][44959] Updated weights for policy 1, policy_version 67920 (0.0008) [2023-10-12 22:51:52,360][44959] Updated weights for policy 1, policy_version 67930 (0.0007) [2023-10-12 22:51:54,399][44958] Updated weights for policy 0, policy_version 67590 (0.0007) [2023-10-12 22:51:54,774][44958] Updated weights for policy 0, policy_version 67600 (0.0007) [2023-10-12 22:51:55,152][44958] Updated weights for policy 0, policy_version 67610 (0.0009) [2023-10-12 22:51:56,425][44959] Updated weights for policy 1, policy_version 67940 (0.0008) [2023-10-12 22:51:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138805248. Throughput: 0: 1641.0, 1: 1650.1. Samples: 34710226. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-12 22:51:56,443][43579] Avg episode reward: [(0, '281.900'), (1, '271.300')] [2023-10-12 22:51:56,791][44959] Updated weights for policy 1, policy_version 67950 (0.0010) [2023-10-12 22:51:57,162][44959] Updated weights for policy 1, policy_version 67960 (0.0009) [2023-10-12 22:51:59,319][44958] Updated weights for policy 0, policy_version 67620 (0.0008) [2023-10-12 22:51:59,696][44958] Updated weights for policy 0, policy_version 67630 (0.0007) [2023-10-12 22:52:00,064][44958] Updated weights for policy 0, policy_version 67640 (0.0008) [2023-10-12 22:52:01,307][44959] Updated weights for policy 1, policy_version 67970 (0.0007) [2023-10-12 22:52:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138870784. Throughput: 0: 1648.6, 1: 1651.4. Samples: 34730412. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-12 22:52:01,443][43579] Avg episode reward: [(0, '278.560'), (1, '269.140')] [2023-10-12 22:52:01,685][44959] Updated weights for policy 1, policy_version 67980 (0.0011) [2023-10-12 22:52:02,044][44959] Updated weights for policy 1, policy_version 67990 (0.0011) [2023-10-12 22:52:02,421][44959] Updated weights for policy 1, policy_version 68000 (0.0008) [2023-10-12 22:52:04,118][44958] Updated weights for policy 0, policy_version 67650 (0.0009) [2023-10-12 22:52:04,491][44958] Updated weights for policy 0, policy_version 67660 (0.0011) [2023-10-12 22:52:04,865][44958] Updated weights for policy 0, policy_version 67670 (0.0007) [2023-10-12 22:52:05,242][44958] Updated weights for policy 0, policy_version 67680 (0.0007) [2023-10-12 22:52:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 138936320. Throughput: 0: 1649.0, 1: 1652.9. Samples: 34740478. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-12 22:52:06,443][43579] Avg episode reward: [(0, '279.220'), (1, '269.110')] [2023-10-12 22:52:06,513][44959] Updated weights for policy 1, policy_version 68010 (0.0010) [2023-10-12 22:52:06,877][44959] Updated weights for policy 1, policy_version 68020 (0.0009) [2023-10-12 22:52:07,242][44959] Updated weights for policy 1, policy_version 68030 (0.0008) [2023-10-12 22:52:09,460][44958] Updated weights for policy 0, policy_version 67690 (0.0009) [2023-10-12 22:52:09,841][44958] Updated weights for policy 0, policy_version 67700 (0.0009) [2023-10-12 22:52:10,220][44958] Updated weights for policy 0, policy_version 67710 (0.0009) [2023-10-12 22:52:11,229][44959] Updated weights for policy 1, policy_version 68040 (0.0008) [2023-10-12 22:52:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139001856. Throughput: 0: 1639.3, 1: 1664.9. Samples: 34759976. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-12 22:52:11,444][43579] Avg episode reward: [(0, '280.580'), (1, '262.730')] [2023-10-12 22:52:11,602][44959] Updated weights for policy 1, policy_version 68050 (0.0009) [2023-10-12 22:52:11,970][44959] Updated weights for policy 1, policy_version 68060 (0.0009) [2023-10-12 22:52:14,266][44958] Updated weights for policy 0, policy_version 67720 (0.0009) [2023-10-12 22:52:14,644][44958] Updated weights for policy 0, policy_version 67730 (0.0009) [2023-10-12 22:52:15,018][44958] Updated weights for policy 0, policy_version 67740 (0.0008) [2023-10-12 22:52:16,060][44959] Updated weights for policy 1, policy_version 68070 (0.0008) [2023-10-12 22:52:16,436][44959] Updated weights for policy 1, policy_version 68080 (0.0008) [2023-10-12 22:52:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139067392. Throughput: 0: 1654.2, 1: 1659.5. Samples: 34780036. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-12 22:52:16,443][43579] Avg episode reward: [(0, '280.520'), (1, '263.370')] [2023-10-12 22:52:16,796][44959] Updated weights for policy 1, policy_version 68090 (0.0007) [2023-10-12 22:52:19,246][44958] Updated weights for policy 0, policy_version 67750 (0.0008) [2023-10-12 22:52:19,620][44958] Updated weights for policy 0, policy_version 67760 (0.0009) [2023-10-12 22:52:19,998][44958] Updated weights for policy 0, policy_version 67770 (0.0007) [2023-10-12 22:52:20,917][44959] Updated weights for policy 1, policy_version 68100 (0.0007) [2023-10-12 22:52:21,288][44959] Updated weights for policy 1, policy_version 68110 (0.0008) [2023-10-12 22:52:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139132928. Throughput: 0: 1650.9, 1: 1662.9. Samples: 34790154. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-12 22:52:21,443][43579] Avg episode reward: [(0, '280.650'), (1, '269.000')] [2023-10-12 22:52:21,647][44959] Updated weights for policy 1, policy_version 68120 (0.0010) [2023-10-12 22:52:24,229][44958] Updated weights for policy 0, policy_version 67780 (0.0008) [2023-10-12 22:52:24,601][44958] Updated weights for policy 0, policy_version 67790 (0.0008) [2023-10-12 22:52:24,970][44958] Updated weights for policy 0, policy_version 67800 (0.0011) [2023-10-12 22:52:25,850][44959] Updated weights for policy 1, policy_version 68130 (0.0010) [2023-10-12 22:52:26,226][44959] Updated weights for policy 1, policy_version 68140 (0.0010) [2023-10-12 22:52:26,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139198464. Throughput: 0: 1643.9, 1: 1668.3. Samples: 34809610. Policy #0 lag: (min: 10.0, avg: 35.0, max: 40.0) [2023-10-12 22:52:26,444][43579] Avg episode reward: [(0, '283.600'), (1, '275.450')] [2023-10-12 22:52:26,597][44959] Updated weights for policy 1, policy_version 68150 (0.0009) [2023-10-12 22:52:26,967][44959] Updated weights for policy 1, policy_version 68160 (0.0010) [2023-10-12 22:52:29,036][44958] Updated weights for policy 0, policy_version 67810 (0.0011) [2023-10-12 22:52:29,413][44958] Updated weights for policy 0, policy_version 67820 (0.0008) [2023-10-12 22:52:29,786][44958] Updated weights for policy 0, policy_version 67830 (0.0009) [2023-10-12 22:52:30,163][44958] Updated weights for policy 0, policy_version 67840 (0.0009) [2023-10-12 22:52:31,046][44959] Updated weights for policy 1, policy_version 68170 (0.0009) [2023-10-12 22:52:31,423][44959] Updated weights for policy 1, policy_version 68180 (0.0009) [2023-10-12 22:52:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139264000. Throughput: 0: 1650.0, 1: 1655.8. Samples: 34829296. Policy #0 lag: (min: 10.0, avg: 35.0, max: 40.0) [2023-10-12 22:52:31,443][43579] Avg episode reward: [(0, '282.220'), (1, '274.840')] [2023-10-12 22:52:31,798][44959] Updated weights for policy 1, policy_version 68190 (0.0009) [2023-10-12 22:52:34,246][44958] Updated weights for policy 0, policy_version 67850 (0.0011) [2023-10-12 22:52:34,631][44958] Updated weights for policy 0, policy_version 67860 (0.0009) [2023-10-12 22:52:35,001][44958] Updated weights for policy 0, policy_version 67870 (0.0008) [2023-10-12 22:52:35,979][44959] Updated weights for policy 1, policy_version 68200 (0.0008) [2023-10-12 22:52:36,343][44959] Updated weights for policy 1, policy_version 68210 (0.0008) [2023-10-12 22:52:36,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139329536. Throughput: 0: 1640.3, 1: 1664.0. Samples: 34839442. Policy #0 lag: (min: 10.0, avg: 35.0, max: 40.0) [2023-10-12 22:52:36,443][43579] Avg episode reward: [(0, '284.100'), (1, '277.750')] [2023-10-12 22:52:36,717][44959] Updated weights for policy 1, policy_version 68220 (0.0008) [2023-10-12 22:52:39,214][44958] Updated weights for policy 0, policy_version 67880 (0.0010) [2023-10-12 22:52:39,603][44958] Updated weights for policy 0, policy_version 67890 (0.0010) [2023-10-12 22:52:39,969][44958] Updated weights for policy 0, policy_version 67900 (0.0011) [2023-10-12 22:52:41,128][44959] Updated weights for policy 1, policy_version 68230 (0.0007) [2023-10-12 22:52:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139395072. Throughput: 0: 1638.4, 1: 1665.6. Samples: 34858906. Policy #0 lag: (min: 10.0, avg: 35.0, max: 40.0) [2023-10-12 22:52:41,443][43579] Avg episode reward: [(0, '280.500'), (1, '279.590')] [2023-10-12 22:52:41,529][44959] Updated weights for policy 1, policy_version 68240 (0.0010) [2023-10-12 22:52:41,900][44959] Updated weights for policy 1, policy_version 68250 (0.0009) [2023-10-12 22:52:44,087][44958] Updated weights for policy 0, policy_version 67910 (0.0009) [2023-10-12 22:52:44,465][44958] Updated weights for policy 0, policy_version 67920 (0.0008) [2023-10-12 22:52:44,844][44958] Updated weights for policy 0, policy_version 67930 (0.0009) [2023-10-12 22:52:45,924][44959] Updated weights for policy 1, policy_version 68260 (0.0009) [2023-10-12 22:52:46,283][44959] Updated weights for policy 1, policy_version 68270 (0.0008) [2023-10-12 22:52:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 139460608. Throughput: 0: 1643.4, 1: 1652.6. Samples: 34878732. Policy #0 lag: (min: 10.0, avg: 35.0, max: 40.0) [2023-10-12 22:52:46,443][43579] Avg episode reward: [(0, '278.860'), (1, '275.670')] [2023-10-12 22:52:46,648][44959] Updated weights for policy 1, policy_version 68280 (0.0008) [2023-10-12 22:52:49,171][44958] Updated weights for policy 0, policy_version 67940 (0.0009) [2023-10-12 22:52:49,548][44958] Updated weights for policy 0, policy_version 67950 (0.0007) [2023-10-12 22:52:49,916][44958] Updated weights for policy 0, policy_version 67960 (0.0009) [2023-10-12 22:52:50,848][44959] Updated weights for policy 1, policy_version 68290 (0.0007) [2023-10-12 22:52:51,212][44959] Updated weights for policy 1, policy_version 68300 (0.0007) [2023-10-12 22:52:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139526144. Throughput: 0: 1638.8, 1: 1655.0. Samples: 34888702. Policy #0 lag: (min: 10.0, avg: 35.0, max: 40.0) [2023-10-12 22:52:51,443][43579] Avg episode reward: [(0, '277.220'), (1, '275.970')] [2023-10-12 22:52:51,582][44959] Updated weights for policy 1, policy_version 68310 (0.0009) [2023-10-12 22:52:51,948][44959] Updated weights for policy 1, policy_version 68320 (0.0007) [2023-10-12 22:52:54,016][44958] Updated weights for policy 0, policy_version 67970 (0.0010) [2023-10-12 22:52:54,395][44958] Updated weights for policy 0, policy_version 67980 (0.0009) [2023-10-12 22:52:54,766][44958] Updated weights for policy 0, policy_version 67990 (0.0008) [2023-10-12 22:52:55,138][44958] Updated weights for policy 0, policy_version 68000 (0.0008) [2023-10-12 22:52:56,128][44959] Updated weights for policy 1, policy_version 68330 (0.0007) [2023-10-12 22:52:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139591680. Throughput: 0: 1642.1, 1: 1647.5. Samples: 34908006. Policy #0 lag: (min: 10.0, avg: 35.0, max: 40.0) [2023-10-12 22:52:56,443][43579] Avg episode reward: [(0, '278.330'), (1, '274.730')] [2023-10-12 22:52:56,490][44959] Updated weights for policy 1, policy_version 68340 (0.0009) [2023-10-12 22:52:56,863][44959] Updated weights for policy 1, policy_version 68350 (0.0010) [2023-10-12 22:52:59,301][44958] Updated weights for policy 0, policy_version 68010 (0.0008) [2023-10-12 22:52:59,670][44958] Updated weights for policy 0, policy_version 68020 (0.0008) [2023-10-12 22:53:00,043][44958] Updated weights for policy 0, policy_version 68030 (0.0008) [2023-10-12 22:53:01,030][44959] Updated weights for policy 1, policy_version 68360 (0.0008) [2023-10-12 22:53:01,406][44959] Updated weights for policy 1, policy_version 68370 (0.0008) [2023-10-12 22:53:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139657216. Throughput: 0: 1644.2, 1: 1641.4. Samples: 34927886. Policy #0 lag: (min: 10.0, avg: 35.0, max: 40.0) [2023-10-12 22:53:01,443][43579] Avg episode reward: [(0, '278.340'), (1, '273.660')] [2023-10-12 22:53:01,767][44959] Updated weights for policy 1, policy_version 68380 (0.0008) [2023-10-12 22:53:04,174][44958] Updated weights for policy 0, policy_version 68040 (0.0008) [2023-10-12 22:53:04,546][44958] Updated weights for policy 0, policy_version 68050 (0.0009) [2023-10-12 22:53:04,926][44958] Updated weights for policy 0, policy_version 68060 (0.0007) [2023-10-12 22:53:05,848][44959] Updated weights for policy 1, policy_version 68390 (0.0008) [2023-10-12 22:53:06,219][44959] Updated weights for policy 1, policy_version 68400 (0.0009) [2023-10-12 22:53:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 139722752. Throughput: 0: 1640.2, 1: 1645.2. Samples: 34937994. Policy #0 lag: (min: 10.0, avg: 35.0, max: 40.0) [2023-10-12 22:53:06,444][43579] Avg episode reward: [(0, '268.790'), (1, '277.190')] [2023-10-12 22:53:06,588][44959] Updated weights for policy 1, policy_version 68410 (0.0010) [2023-10-12 22:53:09,088][44958] Updated weights for policy 0, policy_version 68070 (0.0009) [2023-10-12 22:53:09,456][44958] Updated weights for policy 0, policy_version 68080 (0.0007) [2023-10-12 22:53:09,832][44958] Updated weights for policy 0, policy_version 68090 (0.0007) [2023-10-12 22:53:10,768][44959] Updated weights for policy 1, policy_version 68420 (0.0007) [2023-10-12 22:53:11,127][44959] Updated weights for policy 1, policy_version 68430 (0.0007) [2023-10-12 22:53:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 139788288. Throughput: 0: 1641.3, 1: 1645.0. Samples: 34957494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:11,443][43579] Avg episode reward: [(0, '273.100'), (1, '281.330')] [2023-10-12 22:53:11,499][44959] Updated weights for policy 1, policy_version 68440 (0.0008) [2023-10-12 22:53:13,792][44958] Updated weights for policy 0, policy_version 68100 (0.0008) [2023-10-12 22:53:14,159][44958] Updated weights for policy 0, policy_version 68110 (0.0009) [2023-10-12 22:53:14,524][44958] Updated weights for policy 0, policy_version 68120 (0.0007) [2023-10-12 22:53:15,639][44959] Updated weights for policy 1, policy_version 68450 (0.0008) [2023-10-12 22:53:16,008][44959] Updated weights for policy 1, policy_version 68460 (0.0007) [2023-10-12 22:53:16,373][44959] Updated weights for policy 1, policy_version 68470 (0.0009) [2023-10-12 22:53:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 139853824. Throughput: 0: 1648.0, 1: 1645.0. Samples: 34977480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:16,443][43579] Avg episode reward: [(0, '263.190'), (1, '286.030')] [2023-10-12 22:53:16,742][44959] Updated weights for policy 1, policy_version 68480 (0.0008) [2023-10-12 22:53:18,758][44958] Updated weights for policy 0, policy_version 68130 (0.0008) [2023-10-12 22:53:19,130][44958] Updated weights for policy 0, policy_version 68140 (0.0011) [2023-10-12 22:53:19,500][44958] Updated weights for policy 0, policy_version 68150 (0.0010) [2023-10-12 22:53:19,874][44958] Updated weights for policy 0, policy_version 68160 (0.0008) [2023-10-12 22:53:20,936][44959] Updated weights for policy 1, policy_version 68490 (0.0009) [2023-10-12 22:53:21,307][44959] Updated weights for policy 1, policy_version 68500 (0.0009) [2023-10-12 22:53:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139919360. Throughput: 0: 1643.8, 1: 1650.0. Samples: 34987660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:21,443][43579] Avg episode reward: [(0, '258.650'), (1, '279.330')] [2023-10-12 22:53:21,679][44959] Updated weights for policy 1, policy_version 68510 (0.0009) [2023-10-12 22:53:24,021][44958] Updated weights for policy 0, policy_version 68170 (0.0007) [2023-10-12 22:53:24,400][44958] Updated weights for policy 0, policy_version 68180 (0.0008) [2023-10-12 22:53:24,782][44958] Updated weights for policy 0, policy_version 68190 (0.0008) [2023-10-12 22:53:25,730][44959] Updated weights for policy 1, policy_version 68520 (0.0007) [2023-10-12 22:53:26,111][44959] Updated weights for policy 1, policy_version 68530 (0.0008) [2023-10-12 22:53:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 139984896. Throughput: 0: 1647.9, 1: 1652.1. Samples: 35007406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:26,444][43579] Avg episode reward: [(0, '249.610'), (1, '279.930')] [2023-10-12 22:53:26,481][44959] Updated weights for policy 1, policy_version 68540 (0.0010) [2023-10-12 22:53:28,881][44958] Updated weights for policy 0, policy_version 68200 (0.0008) [2023-10-12 22:53:29,253][44958] Updated weights for policy 0, policy_version 68210 (0.0009) [2023-10-12 22:53:29,638][44958] Updated weights for policy 0, policy_version 68220 (0.0009) [2023-10-12 22:53:30,533][44959] Updated weights for policy 1, policy_version 68550 (0.0010) [2023-10-12 22:53:30,901][44959] Updated weights for policy 1, policy_version 68560 (0.0009) [2023-10-12 22:53:31,274][44959] Updated weights for policy 1, policy_version 68570 (0.0008) [2023-10-12 22:53:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 140050432. Throughput: 0: 1646.4, 1: 1644.0. Samples: 35026798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:31,443][43579] Avg episode reward: [(0, '251.960'), (1, '280.490')] [2023-10-12 22:53:33,928][44958] Updated weights for policy 0, policy_version 68230 (0.0009) [2023-10-12 22:53:34,291][44958] Updated weights for policy 0, policy_version 68240 (0.0007) [2023-10-12 22:53:34,666][44958] Updated weights for policy 0, policy_version 68250 (0.0008) [2023-10-12 22:53:35,402][44959] Updated weights for policy 1, policy_version 68580 (0.0007) [2023-10-12 22:53:35,773][44959] Updated weights for policy 1, policy_version 68590 (0.0007) [2023-10-12 22:53:36,137][44959] Updated weights for policy 1, policy_version 68600 (0.0008) [2023-10-12 22:53:36,442][43579] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 140148736. Throughput: 0: 1641.0, 1: 1657.2. Samples: 35037120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:36,443][43579] Avg episode reward: [(0, '257.540'), (1, '283.250')] [2023-10-12 22:53:38,821][44958] Updated weights for policy 0, policy_version 68260 (0.0007) [2023-10-12 22:53:39,191][44958] Updated weights for policy 0, policy_version 68270 (0.0009) [2023-10-12 22:53:39,568][44958] Updated weights for policy 0, policy_version 68280 (0.0009) [2023-10-12 22:53:40,358][44959] Updated weights for policy 1, policy_version 68610 (0.0008) [2023-10-12 22:53:40,729][44959] Updated weights for policy 1, policy_version 68620 (0.0008) [2023-10-12 22:53:41,103][44959] Updated weights for policy 1, policy_version 68630 (0.0008) [2023-10-12 22:53:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 140181504. Throughput: 0: 1644.9, 1: 1658.0. Samples: 35056638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:41,443][43579] Avg episode reward: [(0, '259.030'), (1, '279.560')] [2023-10-12 22:53:41,470][44959] Updated weights for policy 1, policy_version 68640 (0.0009) [2023-10-12 22:53:43,827][44958] Updated weights for policy 0, policy_version 68290 (0.0007) [2023-10-12 22:53:44,204][44958] Updated weights for policy 0, policy_version 68300 (0.0008) [2023-10-12 22:53:44,580][44958] Updated weights for policy 0, policy_version 68310 (0.0009) [2023-10-12 22:53:44,955][44958] Updated weights for policy 0, policy_version 68320 (0.0010) [2023-10-12 22:53:45,352][44959] Updated weights for policy 1, policy_version 68650 (0.0008) [2023-10-12 22:53:45,722][44959] Updated weights for policy 1, policy_version 68660 (0.0007) [2023-10-12 22:53:46,083][44959] Updated weights for policy 1, policy_version 68670 (0.0009) [2023-10-12 22:53:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 140279808. Throughput: 0: 1644.4, 1: 1649.2. Samples: 35076096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:46,443][43579] Avg episode reward: [(0, '266.410'), (1, '284.130')] [2023-10-12 22:53:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000068672_70320128.pth... [2023-10-12 22:53:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000068320_69959680.pth... [2023-10-12 22:53:46,482][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000067104_68714496.pth [2023-10-12 22:53:46,485][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000066784_68386816.pth [2023-10-12 22:53:49,255][44958] Updated weights for policy 0, policy_version 68330 (0.0008) [2023-10-12 22:53:49,622][44958] Updated weights for policy 0, policy_version 68340 (0.0008) [2023-10-12 22:53:49,998][44958] Updated weights for policy 0, policy_version 68350 (0.0009) [2023-10-12 22:53:50,245][44959] Updated weights for policy 1, policy_version 68680 (0.0009) [2023-10-12 22:53:50,621][44959] Updated weights for policy 1, policy_version 68690 (0.0010) [2023-10-12 22:53:50,980][44959] Updated weights for policy 1, policy_version 68700 (0.0008) [2023-10-12 22:53:51,443][43579] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 140345344. Throughput: 0: 1642.4, 1: 1664.8. Samples: 35086814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:51,444][43579] Avg episode reward: [(0, '274.410'), (1, '286.440')] [2023-10-12 22:53:53,922][44958] Updated weights for policy 0, policy_version 68360 (0.0008) [2023-10-12 22:53:54,289][44958] Updated weights for policy 0, policy_version 68370 (0.0008) [2023-10-12 22:53:54,659][44958] Updated weights for policy 0, policy_version 68380 (0.0008) [2023-10-12 22:53:55,245][44959] Updated weights for policy 1, policy_version 68710 (0.0010) [2023-10-12 22:53:55,602][44959] Updated weights for policy 1, policy_version 68720 (0.0010) [2023-10-12 22:53:55,970][44959] Updated weights for policy 1, policy_version 68730 (0.0008) [2023-10-12 22:53:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 140410880. Throughput: 0: 1647.5, 1: 1659.3. Samples: 35106298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:53:56,443][43579] Avg episode reward: [(0, '280.220'), (1, '286.540')] [2023-10-12 22:53:58,960][44958] Updated weights for policy 0, policy_version 68390 (0.0008) [2023-10-12 22:53:59,343][44958] Updated weights for policy 0, policy_version 68400 (0.0009) [2023-10-12 22:53:59,710][44958] Updated weights for policy 0, policy_version 68410 (0.0008) [2023-10-12 22:54:00,179][44959] Updated weights for policy 1, policy_version 68740 (0.0008) [2023-10-12 22:54:00,557][44959] Updated weights for policy 1, policy_version 68750 (0.0008) [2023-10-12 22:54:00,921][44959] Updated weights for policy 1, policy_version 68760 (0.0007) [2023-10-12 22:54:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 140476416. Throughput: 0: 1639.5, 1: 1647.5. Samples: 35125394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:54:01,444][43579] Avg episode reward: [(0, '276.510'), (1, '291.120')] [2023-10-12 22:54:03,868][44958] Updated weights for policy 0, policy_version 68420 (0.0011) [2023-10-12 22:54:04,241][44958] Updated weights for policy 0, policy_version 68430 (0.0010) [2023-10-12 22:54:04,612][44958] Updated weights for policy 0, policy_version 68440 (0.0007) [2023-10-12 22:54:05,124][44959] Updated weights for policy 1, policy_version 68770 (0.0009) [2023-10-12 22:54:05,490][44959] Updated weights for policy 1, policy_version 68780 (0.0007) [2023-10-12 22:54:05,871][44959] Updated weights for policy 1, policy_version 68790 (0.0008) [2023-10-12 22:54:06,238][44959] Updated weights for policy 1, policy_version 68800 (0.0008) [2023-10-12 22:54:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 140541952. Throughput: 0: 1641.3, 1: 1655.2. Samples: 35136002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:54:06,443][43579] Avg episode reward: [(0, '281.700'), (1, '284.490')] [2023-10-12 22:54:08,745][44958] Updated weights for policy 0, policy_version 68450 (0.0008) [2023-10-12 22:54:09,123][44958] Updated weights for policy 0, policy_version 68460 (0.0011) [2023-10-12 22:54:09,500][44958] Updated weights for policy 0, policy_version 68470 (0.0008) [2023-10-12 22:54:09,868][44958] Updated weights for policy 0, policy_version 68480 (0.0007) [2023-10-12 22:54:10,450][44959] Updated weights for policy 1, policy_version 68810 (0.0008) [2023-10-12 22:54:10,820][44959] Updated weights for policy 1, policy_version 68820 (0.0007) [2023-10-12 22:54:11,194][44959] Updated weights for policy 1, policy_version 68830 (0.0008) [2023-10-12 22:54:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 140607488. Throughput: 0: 1636.5, 1: 1656.2. Samples: 35155580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:54:11,443][43579] Avg episode reward: [(0, '282.230'), (1, '280.780')] [2023-10-12 22:54:14,046][44958] Updated weights for policy 0, policy_version 68490 (0.0009) [2023-10-12 22:54:14,425][44958] Updated weights for policy 0, policy_version 68500 (0.0009) [2023-10-12 22:54:14,796][44958] Updated weights for policy 0, policy_version 68510 (0.0011) [2023-10-12 22:54:15,204][44959] Updated weights for policy 1, policy_version 68840 (0.0008) [2023-10-12 22:54:15,586][44959] Updated weights for policy 1, policy_version 68850 (0.0008) [2023-10-12 22:54:15,954][44959] Updated weights for policy 1, policy_version 68860 (0.0010) [2023-10-12 22:54:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 140673024. Throughput: 0: 1637.7, 1: 1646.7. Samples: 35174594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:54:16,444][43579] Avg episode reward: [(0, '276.670'), (1, '280.690')] [2023-10-12 22:54:19,034][44958] Updated weights for policy 0, policy_version 68520 (0.0008) [2023-10-12 22:54:19,409][44958] Updated weights for policy 0, policy_version 68530 (0.0007) [2023-10-12 22:54:19,780][44958] Updated weights for policy 0, policy_version 68540 (0.0008) [2023-10-12 22:54:20,201][44959] Updated weights for policy 1, policy_version 68870 (0.0009) [2023-10-12 22:54:20,567][44959] Updated weights for policy 1, policy_version 68880 (0.0007) [2023-10-12 22:54:20,930][44959] Updated weights for policy 1, policy_version 68890 (0.0010) [2023-10-12 22:54:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 140738560. Throughput: 0: 1637.3, 1: 1651.4. Samples: 35185112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:54:21,444][43579] Avg episode reward: [(0, '280.190'), (1, '276.710')] [2023-10-12 22:54:23,792][44958] Updated weights for policy 0, policy_version 68550 (0.0007) [2023-10-12 22:54:24,164][44958] Updated weights for policy 0, policy_version 68560 (0.0008) [2023-10-12 22:54:24,549][44958] Updated weights for policy 0, policy_version 68570 (0.0010) [2023-10-12 22:54:25,182][44959] Updated weights for policy 1, policy_version 68900 (0.0009) [2023-10-12 22:54:25,550][44959] Updated weights for policy 1, policy_version 68910 (0.0008) [2023-10-12 22:54:25,928][44959] Updated weights for policy 1, policy_version 68920 (0.0008) [2023-10-12 22:54:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 140804096. Throughput: 0: 1636.4, 1: 1649.4. Samples: 35204500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:54:26,444][43579] Avg episode reward: [(0, '278.300'), (1, '275.750')] [2023-10-12 22:54:28,724][44958] Updated weights for policy 0, policy_version 68580 (0.0007) [2023-10-12 22:54:29,090][44958] Updated weights for policy 0, policy_version 68590 (0.0007) [2023-10-12 22:54:29,462][44958] Updated weights for policy 0, policy_version 68600 (0.0007) [2023-10-12 22:54:30,019][44959] Updated weights for policy 1, policy_version 68930 (0.0009) [2023-10-12 22:54:30,383][44959] Updated weights for policy 1, policy_version 68940 (0.0009) [2023-10-12 22:54:30,750][44959] Updated weights for policy 1, policy_version 68950 (0.0008) [2023-10-12 22:54:31,117][44959] Updated weights for policy 1, policy_version 68960 (0.0009) [2023-10-12 22:54:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 140869632. Throughput: 0: 1637.2, 1: 1640.4. Samples: 35223586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:54:31,443][43579] Avg episode reward: [(0, '276.510'), (1, '273.840')] [2023-10-12 22:54:33,755][44958] Updated weights for policy 0, policy_version 68610 (0.0008) [2023-10-12 22:54:34,174][44958] Updated weights for policy 0, policy_version 68620 (0.0009) [2023-10-12 22:54:34,540][44958] Updated weights for policy 0, policy_version 68630 (0.0008) [2023-10-12 22:54:34,913][44958] Updated weights for policy 0, policy_version 68640 (0.0008) [2023-10-12 22:54:35,265][44959] Updated weights for policy 1, policy_version 68970 (0.0007) [2023-10-12 22:54:35,638][44959] Updated weights for policy 1, policy_version 68980 (0.0008) [2023-10-12 22:54:36,005][44959] Updated weights for policy 1, policy_version 68990 (0.0007) [2023-10-12 22:54:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 140935168. Throughput: 0: 1632.5, 1: 1640.3. Samples: 35234090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:54:36,443][43579] Avg episode reward: [(0, '269.150'), (1, '276.340')] [2023-10-12 22:54:39,102][44958] Updated weights for policy 0, policy_version 68650 (0.0007) [2023-10-12 22:54:39,483][44958] Updated weights for policy 0, policy_version 68660 (0.0008) [2023-10-12 22:54:39,856][44958] Updated weights for policy 0, policy_version 68670 (0.0011) [2023-10-12 22:54:40,159][44959] Updated weights for policy 1, policy_version 69000 (0.0010) [2023-10-12 22:54:40,517][44959] Updated weights for policy 1, policy_version 69010 (0.0007) [2023-10-12 22:54:40,887][44959] Updated weights for policy 1, policy_version 69020 (0.0010) [2023-10-12 22:54:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 141000704. Throughput: 0: 1631.3, 1: 1637.4. Samples: 35253392. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-12 22:54:41,443][43579] Avg episode reward: [(0, '275.590'), (1, '279.230')] [2023-10-12 22:54:43,832][44958] Updated weights for policy 0, policy_version 68680 (0.0008) [2023-10-12 22:54:44,196][44958] Updated weights for policy 0, policy_version 68690 (0.0009) [2023-10-12 22:54:44,578][44958] Updated weights for policy 0, policy_version 68700 (0.0009) [2023-10-12 22:54:44,984][44959] Updated weights for policy 1, policy_version 69030 (0.0009) [2023-10-12 22:54:45,359][44959] Updated weights for policy 1, policy_version 69040 (0.0010) [2023-10-12 22:54:45,727][44959] Updated weights for policy 1, policy_version 69050 (0.0009) [2023-10-12 22:54:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141066240. Throughput: 0: 1641.0, 1: 1639.8. Samples: 35273030. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-12 22:54:46,443][43579] Avg episode reward: [(0, '274.250'), (1, '276.880')] [2023-10-12 22:54:48,777][44958] Updated weights for policy 0, policy_version 68710 (0.0008) [2023-10-12 22:54:49,149][44958] Updated weights for policy 0, policy_version 68720 (0.0010) [2023-10-12 22:54:49,521][44958] Updated weights for policy 0, policy_version 68730 (0.0008) [2023-10-12 22:54:50,030][44959] Updated weights for policy 1, policy_version 69060 (0.0008) [2023-10-12 22:54:50,390][44959] Updated weights for policy 1, policy_version 69070 (0.0008) [2023-10-12 22:54:50,759][44959] Updated weights for policy 1, policy_version 69080 (0.0007) [2023-10-12 22:54:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141131776. Throughput: 0: 1634.3, 1: 1647.1. Samples: 35283666. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-12 22:54:51,444][43579] Avg episode reward: [(0, '264.640'), (1, '270.420')] [2023-10-12 22:54:53,809][44958] Updated weights for policy 0, policy_version 68740 (0.0007) [2023-10-12 22:54:54,172][44958] Updated weights for policy 0, policy_version 68750 (0.0007) [2023-10-12 22:54:54,545][44958] Updated weights for policy 0, policy_version 68760 (0.0007) [2023-10-12 22:54:55,085][44959] Updated weights for policy 1, policy_version 69090 (0.0008) [2023-10-12 22:54:55,465][44959] Updated weights for policy 1, policy_version 69100 (0.0010) [2023-10-12 22:54:55,835][44959] Updated weights for policy 1, policy_version 69110 (0.0008) [2023-10-12 22:54:56,207][44959] Updated weights for policy 1, policy_version 69120 (0.0008) [2023-10-12 22:54:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141197312. Throughput: 0: 1638.6, 1: 1639.2. Samples: 35303082. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-12 22:54:56,444][43579] Avg episode reward: [(0, '263.790'), (1, '270.810')] [2023-10-12 22:54:58,906][44958] Updated weights for policy 0, policy_version 68770 (0.0008) [2023-10-12 22:54:59,273][44958] Updated weights for policy 0, policy_version 68780 (0.0009) [2023-10-12 22:54:59,644][44958] Updated weights for policy 0, policy_version 68790 (0.0010) [2023-10-12 22:55:00,018][44958] Updated weights for policy 0, policy_version 68800 (0.0009) [2023-10-12 22:55:00,477][44959] Updated weights for policy 1, policy_version 69130 (0.0008) [2023-10-12 22:55:00,846][44959] Updated weights for policy 1, policy_version 69140 (0.0008) [2023-10-12 22:55:01,216][44959] Updated weights for policy 1, policy_version 69150 (0.0009) [2023-10-12 22:55:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141262848. Throughput: 0: 1638.3, 1: 1641.2. Samples: 35322168. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-12 22:55:01,443][43579] Avg episode reward: [(0, '267.300'), (1, '271.730')] [2023-10-12 22:55:03,971][44958] Updated weights for policy 0, policy_version 68810 (0.0009) [2023-10-12 22:55:04,343][44958] Updated weights for policy 0, policy_version 68820 (0.0010) [2023-10-12 22:55:04,713][44958] Updated weights for policy 0, policy_version 68830 (0.0008) [2023-10-12 22:55:05,189][44959] Updated weights for policy 1, policy_version 69160 (0.0009) [2023-10-12 22:55:05,548][44959] Updated weights for policy 1, policy_version 69170 (0.0007) [2023-10-12 22:55:05,912][44959] Updated weights for policy 1, policy_version 69180 (0.0007) [2023-10-12 22:55:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141328384. Throughput: 0: 1640.9, 1: 1638.8. Samples: 35332696. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-12 22:55:06,443][43579] Avg episode reward: [(0, '268.370'), (1, '272.650')] [2023-10-12 22:55:08,939][44958] Updated weights for policy 0, policy_version 68840 (0.0008) [2023-10-12 22:55:09,315][44958] Updated weights for policy 0, policy_version 68850 (0.0007) [2023-10-12 22:55:09,687][44958] Updated weights for policy 0, policy_version 68860 (0.0007) [2023-10-12 22:55:10,077][44959] Updated weights for policy 1, policy_version 69190 (0.0008) [2023-10-12 22:55:10,442][44959] Updated weights for policy 1, policy_version 69200 (0.0009) [2023-10-12 22:55:10,805][44959] Updated weights for policy 1, policy_version 69210 (0.0007) [2023-10-12 22:55:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141393920. Throughput: 0: 1642.4, 1: 1640.2. Samples: 35352218. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-12 22:55:11,444][43579] Avg episode reward: [(0, '267.990'), (1, '270.240')] [2023-10-12 22:55:13,740][44958] Updated weights for policy 0, policy_version 68870 (0.0008) [2023-10-12 22:55:14,117][44958] Updated weights for policy 0, policy_version 68880 (0.0009) [2023-10-12 22:55:14,491][44958] Updated weights for policy 0, policy_version 68890 (0.0007) [2023-10-12 22:55:14,873][44959] Updated weights for policy 1, policy_version 69220 (0.0007) [2023-10-12 22:55:15,235][44959] Updated weights for policy 1, policy_version 69230 (0.0011) [2023-10-12 22:55:15,602][44959] Updated weights for policy 1, policy_version 69240 (0.0009) [2023-10-12 22:55:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141459456. Throughput: 0: 1649.1, 1: 1644.0. Samples: 35371772. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-12 22:55:16,444][43579] Avg episode reward: [(0, '271.150'), (1, '275.920')] [2023-10-12 22:55:18,727][44958] Updated weights for policy 0, policy_version 68900 (0.0008) [2023-10-12 22:55:19,109][44958] Updated weights for policy 0, policy_version 68910 (0.0008) [2023-10-12 22:55:19,482][44958] Updated weights for policy 0, policy_version 68920 (0.0008) [2023-10-12 22:55:19,782][44959] Updated weights for policy 1, policy_version 69250 (0.0010) [2023-10-12 22:55:20,157][44959] Updated weights for policy 1, policy_version 69260 (0.0008) [2023-10-12 22:55:20,522][44959] Updated weights for policy 1, policy_version 69270 (0.0007) [2023-10-12 22:55:20,894][44959] Updated weights for policy 1, policy_version 69280 (0.0007) [2023-10-12 22:55:21,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141524992. Throughput: 0: 1649.2, 1: 1648.7. Samples: 35382492. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-12 22:55:21,443][43579] Avg episode reward: [(0, '278.340'), (1, '280.910')] [2023-10-12 22:55:23,726][44958] Updated weights for policy 0, policy_version 68930 (0.0008) [2023-10-12 22:55:24,098][44958] Updated weights for policy 0, policy_version 68940 (0.0008) [2023-10-12 22:55:24,477][44958] Updated weights for policy 0, policy_version 68950 (0.0007) [2023-10-12 22:55:24,844][44958] Updated weights for policy 0, policy_version 68960 (0.0008) [2023-10-12 22:55:25,011][44959] Updated weights for policy 1, policy_version 69290 (0.0007) [2023-10-12 22:55:25,373][44959] Updated weights for policy 1, policy_version 69300 (0.0011) [2023-10-12 22:55:25,747][44959] Updated weights for policy 1, policy_version 69310 (0.0009) [2023-10-12 22:55:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141590528. Throughput: 0: 1648.2, 1: 1644.5. Samples: 35401564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 22:55:26,444][43579] Avg episode reward: [(0, '271.300'), (1, '275.760')] [2023-10-12 22:55:29,039][44958] Updated weights for policy 0, policy_version 68970 (0.0007) [2023-10-12 22:55:29,407][44958] Updated weights for policy 0, policy_version 68980 (0.0010) [2023-10-12 22:55:29,779][44958] Updated weights for policy 0, policy_version 68990 (0.0008) [2023-10-12 22:55:29,883][44959] Updated weights for policy 1, policy_version 69320 (0.0007) [2023-10-12 22:55:30,250][44959] Updated weights for policy 1, policy_version 69330 (0.0010) [2023-10-12 22:55:30,617][44959] Updated weights for policy 1, policy_version 69340 (0.0008) [2023-10-12 22:55:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141656064. Throughput: 0: 1642.1, 1: 1642.5. Samples: 35420840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 22:55:31,443][43579] Avg episode reward: [(0, '268.880'), (1, '276.330')] [2023-10-12 22:55:33,913][44958] Updated weights for policy 0, policy_version 69000 (0.0008) [2023-10-12 22:55:34,289][44958] Updated weights for policy 0, policy_version 69010 (0.0009) [2023-10-12 22:55:34,672][44958] Updated weights for policy 0, policy_version 69020 (0.0009) [2023-10-12 22:55:34,855][44959] Updated weights for policy 1, policy_version 69350 (0.0008) [2023-10-12 22:55:35,216][44959] Updated weights for policy 1, policy_version 69360 (0.0008) [2023-10-12 22:55:35,576][44959] Updated weights for policy 1, policy_version 69370 (0.0008) [2023-10-12 22:55:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141721600. Throughput: 0: 1647.5, 1: 1644.0. Samples: 35431782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 22:55:36,444][43579] Avg episode reward: [(0, '269.420'), (1, '278.740')] [2023-10-12 22:55:38,735][44958] Updated weights for policy 0, policy_version 69030 (0.0007) [2023-10-12 22:55:39,104][44958] Updated weights for policy 0, policy_version 69040 (0.0007) [2023-10-12 22:55:39,479][44958] Updated weights for policy 0, policy_version 69050 (0.0008) [2023-10-12 22:55:39,852][44959] Updated weights for policy 1, policy_version 69380 (0.0008) [2023-10-12 22:55:40,217][44959] Updated weights for policy 1, policy_version 69390 (0.0009) [2023-10-12 22:55:40,576][44959] Updated weights for policy 1, policy_version 69400 (0.0009) [2023-10-12 22:55:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141787136. Throughput: 0: 1645.7, 1: 1639.7. Samples: 35450926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 22:55:41,444][43579] Avg episode reward: [(0, '271.400'), (1, '280.470')] [2023-10-12 22:55:43,744][44958] Updated weights for policy 0, policy_version 69060 (0.0008) [2023-10-12 22:55:44,132][44958] Updated weights for policy 0, policy_version 69070 (0.0009) [2023-10-12 22:55:44,507][44958] Updated weights for policy 0, policy_version 69080 (0.0009) [2023-10-12 22:55:44,547][44959] Updated weights for policy 1, policy_version 69410 (0.0008) [2023-10-12 22:55:44,953][44959] Updated weights for policy 1, policy_version 69420 (0.0009) [2023-10-12 22:55:45,318][44959] Updated weights for policy 1, policy_version 69430 (0.0007) [2023-10-12 22:55:45,680][44959] Updated weights for policy 1, policy_version 69440 (0.0007) [2023-10-12 22:55:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141852672. Throughput: 0: 1647.2, 1: 1650.0. Samples: 35470538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 22:55:46,443][43579] Avg episode reward: [(0, '270.420'), (1, '280.260')] [2023-10-12 22:55:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000069088_70746112.pth... [2023-10-12 22:55:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000069440_71106560.pth... [2023-10-12 22:55:46,482][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000067552_69173248.pth [2023-10-12 22:55:46,486][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000069088_70746112.pth [2023-10-12 22:55:46,491][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000067904_69533696.pth [2023-10-12 22:55:46,497][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000069440_71106560.pth [2023-10-12 22:55:48,539][44958] Updated weights for policy 0, policy_version 69090 (0.0009) [2023-10-12 22:55:48,905][44958] Updated weights for policy 0, policy_version 69100 (0.0008) [2023-10-12 22:55:49,285][44958] Updated weights for policy 0, policy_version 69110 (0.0009) [2023-10-12 22:55:49,656][44958] Updated weights for policy 0, policy_version 69120 (0.0008) [2023-10-12 22:55:49,687][44959] Updated weights for policy 1, policy_version 69450 (0.0009) [2023-10-12 22:55:50,057][44959] Updated weights for policy 1, policy_version 69460 (0.0009) [2023-10-12 22:55:50,432][44959] Updated weights for policy 1, policy_version 69470 (0.0007) [2023-10-12 22:55:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141918208. Throughput: 0: 1642.4, 1: 1660.6. Samples: 35481328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 22:55:51,443][43579] Avg episode reward: [(0, '272.240'), (1, '283.280')] [2023-10-12 22:55:53,828][44958] Updated weights for policy 0, policy_version 69130 (0.0010) [2023-10-12 22:55:54,207][44958] Updated weights for policy 0, policy_version 69140 (0.0010) [2023-10-12 22:55:54,582][44958] Updated weights for policy 0, policy_version 69150 (0.0009) [2023-10-12 22:55:54,714][44959] Updated weights for policy 1, policy_version 69480 (0.0008) [2023-10-12 22:55:55,088][44959] Updated weights for policy 1, policy_version 69490 (0.0008) [2023-10-12 22:55:55,447][44959] Updated weights for policy 1, policy_version 69500 (0.0007) [2023-10-12 22:55:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 141983744. Throughput: 0: 1647.2, 1: 1645.4. Samples: 35500388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 22:55:56,443][43579] Avg episode reward: [(0, '270.960'), (1, '285.530')] [2023-10-12 22:55:58,798][44958] Updated weights for policy 0, policy_version 69160 (0.0009) [2023-10-12 22:55:59,169][44958] Updated weights for policy 0, policy_version 69170 (0.0008) [2023-10-12 22:55:59,545][44958] Updated weights for policy 0, policy_version 69180 (0.0007) [2023-10-12 22:55:59,575][44959] Updated weights for policy 1, policy_version 69510 (0.0008) [2023-10-12 22:55:59,954][44959] Updated weights for policy 1, policy_version 69520 (0.0010) [2023-10-12 22:56:00,319][44959] Updated weights for policy 1, policy_version 69530 (0.0009) [2023-10-12 22:56:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142049280. Throughput: 0: 1641.4, 1: 1652.0. Samples: 35519976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 22:56:01,443][43579] Avg episode reward: [(0, '274.400'), (1, '286.600')] [2023-10-12 22:56:03,561][44958] Updated weights for policy 0, policy_version 69190 (0.0008) [2023-10-12 22:56:03,945][44958] Updated weights for policy 0, policy_version 69200 (0.0010) [2023-10-12 22:56:04,321][44958] Updated weights for policy 0, policy_version 69210 (0.0010) [2023-10-12 22:56:04,426][44959] Updated weights for policy 1, policy_version 69540 (0.0007) [2023-10-12 22:56:04,793][44959] Updated weights for policy 1, policy_version 69550 (0.0009) [2023-10-12 22:56:05,161][44959] Updated weights for policy 1, policy_version 69560 (0.0010) [2023-10-12 22:56:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142114816. Throughput: 0: 1638.9, 1: 1652.5. Samples: 35530604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-12 22:56:06,444][43579] Avg episode reward: [(0, '272.800'), (1, '285.640')] [2023-10-12 22:56:08,610][44958] Updated weights for policy 0, policy_version 69220 (0.0008) [2023-10-12 22:56:08,981][44958] Updated weights for policy 0, policy_version 69230 (0.0007) [2023-10-12 22:56:09,360][44958] Updated weights for policy 0, policy_version 69240 (0.0008) [2023-10-12 22:56:09,437][44959] Updated weights for policy 1, policy_version 69570 (0.0010) [2023-10-12 22:56:09,798][44959] Updated weights for policy 1, policy_version 69580 (0.0007) [2023-10-12 22:56:10,163][44959] Updated weights for policy 1, policy_version 69590 (0.0008) [2023-10-12 22:56:10,534][44959] Updated weights for policy 1, policy_version 69600 (0.0011) [2023-10-12 22:56:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142180352. Throughput: 0: 1644.9, 1: 1645.6. Samples: 35549638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:11,444][43579] Avg episode reward: [(0, '271.750'), (1, '282.380')] [2023-10-12 22:56:13,855][44958] Updated weights for policy 0, policy_version 69250 (0.0009) [2023-10-12 22:56:14,218][44958] Updated weights for policy 0, policy_version 69260 (0.0007) [2023-10-12 22:56:14,509][44959] Updated weights for policy 1, policy_version 69610 (0.0008) [2023-10-12 22:56:14,595][44958] Updated weights for policy 0, policy_version 69270 (0.0007) [2023-10-12 22:56:14,883][44959] Updated weights for policy 1, policy_version 69620 (0.0009) [2023-10-12 22:56:14,970][44958] Updated weights for policy 0, policy_version 69280 (0.0008) [2023-10-12 22:56:15,240][44959] Updated weights for policy 1, policy_version 69630 (0.0009) [2023-10-12 22:56:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 142245888. Throughput: 0: 1643.4, 1: 1659.2. Samples: 35569458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:16,443][43579] Avg episode reward: [(0, '272.520'), (1, '282.530')] [2023-10-12 22:56:18,882][44958] Updated weights for policy 0, policy_version 69290 (0.0007) [2023-10-12 22:56:19,257][44958] Updated weights for policy 0, policy_version 69300 (0.0008) [2023-10-12 22:56:19,480][44959] Updated weights for policy 1, policy_version 69640 (0.0008) [2023-10-12 22:56:19,622][44958] Updated weights for policy 0, policy_version 69310 (0.0007) [2023-10-12 22:56:19,850][44959] Updated weights for policy 1, policy_version 69650 (0.0007) [2023-10-12 22:56:20,228][44959] Updated weights for policy 1, policy_version 69660 (0.0007) [2023-10-12 22:56:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142311424. Throughput: 0: 1640.5, 1: 1657.1. Samples: 35580174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:21,444][43579] Avg episode reward: [(0, '275.010'), (1, '283.150')] [2023-10-12 22:56:23,765][44958] Updated weights for policy 0, policy_version 69320 (0.0008) [2023-10-12 22:56:24,137][44958] Updated weights for policy 0, policy_version 69330 (0.0009) [2023-10-12 22:56:24,357][44959] Updated weights for policy 1, policy_version 69670 (0.0008) [2023-10-12 22:56:24,515][44958] Updated weights for policy 0, policy_version 69340 (0.0009) [2023-10-12 22:56:24,724][44959] Updated weights for policy 1, policy_version 69680 (0.0007) [2023-10-12 22:56:25,095][44959] Updated weights for policy 1, policy_version 69690 (0.0008) [2023-10-12 22:56:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142376960. Throughput: 0: 1638.0, 1: 1646.9. Samples: 35598750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:26,444][43579] Avg episode reward: [(0, '278.660'), (1, '275.110')] [2023-10-12 22:56:28,852][44958] Updated weights for policy 0, policy_version 69350 (0.0007) [2023-10-12 22:56:29,107][44959] Updated weights for policy 1, policy_version 69700 (0.0008) [2023-10-12 22:56:29,231][44958] Updated weights for policy 0, policy_version 69360 (0.0008) [2023-10-12 22:56:29,480][44959] Updated weights for policy 1, policy_version 69710 (0.0007) [2023-10-12 22:56:29,599][44958] Updated weights for policy 0, policy_version 69370 (0.0009) [2023-10-12 22:56:29,843][44959] Updated weights for policy 1, policy_version 69720 (0.0009) [2023-10-12 22:56:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 142442496. Throughput: 0: 1634.3, 1: 1658.8. Samples: 35618732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:31,444][43579] Avg episode reward: [(0, '275.520'), (1, '274.250')] [2023-10-12 22:56:33,762][44958] Updated weights for policy 0, policy_version 69380 (0.0008) [2023-10-12 22:56:33,997][44959] Updated weights for policy 1, policy_version 69730 (0.0009) [2023-10-12 22:56:34,125][44958] Updated weights for policy 0, policy_version 69390 (0.0008) [2023-10-12 22:56:34,403][44959] Updated weights for policy 1, policy_version 69740 (0.0007) [2023-10-12 22:56:34,504][44958] Updated weights for policy 0, policy_version 69400 (0.0007) [2023-10-12 22:56:34,775][44959] Updated weights for policy 1, policy_version 69750 (0.0010) [2023-10-12 22:56:35,138][44959] Updated weights for policy 1, policy_version 69760 (0.0010) [2023-10-12 22:56:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142508032. Throughput: 0: 1636.2, 1: 1654.1. Samples: 35629392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:36,444][43579] Avg episode reward: [(0, '276.820'), (1, '272.380')] [2023-10-12 22:56:38,812][44958] Updated weights for policy 0, policy_version 69410 (0.0008) [2023-10-12 22:56:39,188][44958] Updated weights for policy 0, policy_version 69420 (0.0009) [2023-10-12 22:56:39,467][44959] Updated weights for policy 1, policy_version 69770 (0.0009) [2023-10-12 22:56:39,558][44958] Updated weights for policy 0, policy_version 69430 (0.0007) [2023-10-12 22:56:39,837][44959] Updated weights for policy 1, policy_version 69780 (0.0008) [2023-10-12 22:56:39,932][44958] Updated weights for policy 0, policy_version 69440 (0.0007) [2023-10-12 22:56:40,196][44959] Updated weights for policy 1, policy_version 69790 (0.0010) [2023-10-12 22:56:41,442][43579] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142573568. Throughput: 0: 1631.5, 1: 1645.7. Samples: 35647860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:41,443][43579] Avg episode reward: [(0, '280.800'), (1, '277.540')] [2023-10-12 22:56:44,056][44958] Updated weights for policy 0, policy_version 69450 (0.0008) [2023-10-12 22:56:44,410][44959] Updated weights for policy 1, policy_version 69800 (0.0009) [2023-10-12 22:56:44,438][44958] Updated weights for policy 0, policy_version 69460 (0.0007) [2023-10-12 22:56:44,779][44959] Updated weights for policy 1, policy_version 69810 (0.0007) [2023-10-12 22:56:44,811][44958] Updated weights for policy 0, policy_version 69470 (0.0007) [2023-10-12 22:56:45,145][44959] Updated weights for policy 1, policy_version 69820 (0.0011) [2023-10-12 22:56:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142639104. Throughput: 0: 1637.8, 1: 1650.9. Samples: 35667970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:46,444][43579] Avg episode reward: [(0, '283.900'), (1, '275.470')] [2023-10-12 22:56:49,027][44958] Updated weights for policy 0, policy_version 69480 (0.0007) [2023-10-12 22:56:49,204][44959] Updated weights for policy 1, policy_version 69830 (0.0008) [2023-10-12 22:56:49,396][44958] Updated weights for policy 0, policy_version 69490 (0.0009) [2023-10-12 22:56:49,575][44959] Updated weights for policy 1, policy_version 69840 (0.0008) [2023-10-12 22:56:49,760][44958] Updated weights for policy 0, policy_version 69500 (0.0008) [2023-10-12 22:56:49,941][44959] Updated weights for policy 1, policy_version 69850 (0.0007) [2023-10-12 22:56:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142704640. Throughput: 0: 1643.2, 1: 1649.6. Samples: 35678780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:51,443][43579] Avg episode reward: [(0, '283.410'), (1, '273.090')] [2023-10-12 22:56:54,091][44958] Updated weights for policy 0, policy_version 69510 (0.0008) [2023-10-12 22:56:54,202][44959] Updated weights for policy 1, policy_version 69860 (0.0011) [2023-10-12 22:56:54,459][44958] Updated weights for policy 0, policy_version 69520 (0.0007) [2023-10-12 22:56:54,577][44959] Updated weights for policy 1, policy_version 69870 (0.0007) [2023-10-12 22:56:54,842][44958] Updated weights for policy 0, policy_version 69530 (0.0008) [2023-10-12 22:56:54,943][44959] Updated weights for policy 1, policy_version 69880 (0.0008) [2023-10-12 22:56:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142770176. Throughput: 0: 1627.9, 1: 1645.1. Samples: 35696920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:56:56,444][43579] Avg episode reward: [(0, '280.510'), (1, '277.820')] [2023-10-12 22:56:58,945][44958] Updated weights for policy 0, policy_version 69540 (0.0010) [2023-10-12 22:56:59,057][44959] Updated weights for policy 1, policy_version 69890 (0.0010) [2023-10-12 22:56:59,316][44958] Updated weights for policy 0, policy_version 69550 (0.0009) [2023-10-12 22:56:59,422][44959] Updated weights for policy 1, policy_version 69900 (0.0010) [2023-10-12 22:56:59,689][44958] Updated weights for policy 0, policy_version 69560 (0.0009) [2023-10-12 22:56:59,791][44959] Updated weights for policy 1, policy_version 69910 (0.0007) [2023-10-12 22:57:00,163][44959] Updated weights for policy 1, policy_version 69920 (0.0008) [2023-10-12 22:57:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142835712. Throughput: 0: 1629.9, 1: 1646.6. Samples: 35716898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:57:01,443][43579] Avg episode reward: [(0, '275.400'), (1, '278.540')] [2023-10-12 22:57:04,034][44958] Updated weights for policy 0, policy_version 69570 (0.0009) [2023-10-12 22:57:04,394][44959] Updated weights for policy 1, policy_version 69930 (0.0008) [2023-10-12 22:57:04,396][44958] Updated weights for policy 0, policy_version 69580 (0.0009) [2023-10-12 22:57:04,774][44958] Updated weights for policy 0, policy_version 69590 (0.0008) [2023-10-12 22:57:04,775][44959] Updated weights for policy 1, policy_version 69940 (0.0010) [2023-10-12 22:57:05,138][44958] Updated weights for policy 0, policy_version 69600 (0.0007) [2023-10-12 22:57:05,145][44959] Updated weights for policy 1, policy_version 69950 (0.0008) [2023-10-12 22:57:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142901248. Throughput: 0: 1634.9, 1: 1648.1. Samples: 35727910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:57:06,443][43579] Avg episode reward: [(0, '275.590'), (1, '279.840')] [2023-10-12 22:57:09,340][44958] Updated weights for policy 0, policy_version 69610 (0.0008) [2023-10-12 22:57:09,390][44959] Updated weights for policy 1, policy_version 69960 (0.0008) [2023-10-12 22:57:09,710][44958] Updated weights for policy 0, policy_version 69620 (0.0008) [2023-10-12 22:57:09,749][44959] Updated weights for policy 1, policy_version 69970 (0.0009) [2023-10-12 22:57:10,084][44958] Updated weights for policy 0, policy_version 69630 (0.0009) [2023-10-12 22:57:10,121][44959] Updated weights for policy 1, policy_version 69980 (0.0008) [2023-10-12 22:57:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 142966784. Throughput: 0: 1632.4, 1: 1639.8. Samples: 35746002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:57:11,443][43579] Avg episode reward: [(0, '273.780'), (1, '280.200')] [2023-10-12 22:57:13,951][44958] Updated weights for policy 0, policy_version 69640 (0.0008) [2023-10-12 22:57:14,322][44958] Updated weights for policy 0, policy_version 69650 (0.0009) [2023-10-12 22:57:14,379][44959] Updated weights for policy 1, policy_version 69990 (0.0008) [2023-10-12 22:57:14,685][44958] Updated weights for policy 0, policy_version 69660 (0.0008) [2023-10-12 22:57:14,749][44959] Updated weights for policy 1, policy_version 70000 (0.0008) [2023-10-12 22:57:15,115][44959] Updated weights for policy 1, policy_version 70010 (0.0007) [2023-10-12 22:57:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143032320. Throughput: 0: 1637.1, 1: 1631.3. Samples: 35765808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:57:16,444][43579] Avg episode reward: [(0, '272.840'), (1, '279.230')] [2023-10-12 22:57:19,099][44958] Updated weights for policy 0, policy_version 69670 (0.0009) [2023-10-12 22:57:19,468][44958] Updated weights for policy 0, policy_version 69680 (0.0008) [2023-10-12 22:57:19,523][44959] Updated weights for policy 1, policy_version 70020 (0.0009) [2023-10-12 22:57:19,839][44958] Updated weights for policy 0, policy_version 69690 (0.0007) [2023-10-12 22:57:19,920][44959] Updated weights for policy 1, policy_version 70030 (0.0010) [2023-10-12 22:57:20,284][44959] Updated weights for policy 1, policy_version 70040 (0.0007) [2023-10-12 22:57:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143097856. Throughput: 0: 1640.7, 1: 1632.3. Samples: 35776674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:57:21,444][43579] Avg episode reward: [(0, '274.360'), (1, '284.330')] [2023-10-12 22:57:23,880][44958] Updated weights for policy 0, policy_version 69700 (0.0008) [2023-10-12 22:57:24,209][44959] Updated weights for policy 1, policy_version 70050 (0.0008) [2023-10-12 22:57:24,254][44958] Updated weights for policy 0, policy_version 69710 (0.0008) [2023-10-12 22:57:24,570][44959] Updated weights for policy 1, policy_version 70060 (0.0007) [2023-10-12 22:57:24,624][44958] Updated weights for policy 0, policy_version 69720 (0.0008) [2023-10-12 22:57:24,926][44959] Updated weights for policy 1, policy_version 70070 (0.0008) [2023-10-12 22:57:25,293][44959] Updated weights for policy 1, policy_version 70080 (0.0011) [2023-10-12 22:57:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143163392. Throughput: 0: 1639.5, 1: 1635.9. Samples: 35795250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:57:26,444][43579] Avg episode reward: [(0, '277.060'), (1, '284.970')] [2023-10-12 22:57:28,694][44958] Updated weights for policy 0, policy_version 69730 (0.0009) [2023-10-12 22:57:29,067][44958] Updated weights for policy 0, policy_version 69740 (0.0007) [2023-10-12 22:57:29,431][44958] Updated weights for policy 0, policy_version 69750 (0.0007) [2023-10-12 22:57:29,473][44959] Updated weights for policy 1, policy_version 70090 (0.0008) [2023-10-12 22:57:29,798][44958] Updated weights for policy 0, policy_version 69760 (0.0007) [2023-10-12 22:57:29,838][44959] Updated weights for policy 1, policy_version 70100 (0.0008) [2023-10-12 22:57:30,214][44959] Updated weights for policy 1, policy_version 70110 (0.0008) [2023-10-12 22:57:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 143228928. Throughput: 0: 1634.9, 1: 1640.5. Samples: 35815360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:57:31,443][43579] Avg episode reward: [(0, '281.660'), (1, '286.730')] [2023-10-12 22:57:34,129][44958] Updated weights for policy 0, policy_version 69770 (0.0008) [2023-10-12 22:57:34,416][44959] Updated weights for policy 1, policy_version 70120 (0.0008) [2023-10-12 22:57:34,489][44958] Updated weights for policy 0, policy_version 69780 (0.0007) [2023-10-12 22:57:34,789][44959] Updated weights for policy 1, policy_version 70130 (0.0009) [2023-10-12 22:57:34,870][44958] Updated weights for policy 0, policy_version 69790 (0.0009) [2023-10-12 22:57:35,158][44959] Updated weights for policy 1, policy_version 70140 (0.0007) [2023-10-12 22:57:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143294464. Throughput: 0: 1635.1, 1: 1641.2. Samples: 35826214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 22:57:36,444][43579] Avg episode reward: [(0, '280.810'), (1, '287.660')] [2023-10-12 22:57:38,959][44958] Updated weights for policy 0, policy_version 69800 (0.0008) [2023-10-12 22:57:39,292][44959] Updated weights for policy 1, policy_version 70150 (0.0007) [2023-10-12 22:57:39,337][44958] Updated weights for policy 0, policy_version 69810 (0.0008) [2023-10-12 22:57:39,661][44959] Updated weights for policy 1, policy_version 70160 (0.0007) [2023-10-12 22:57:39,703][44958] Updated weights for policy 0, policy_version 69820 (0.0007) [2023-10-12 22:57:40,024][44959] Updated weights for policy 1, policy_version 70170 (0.0009) [2023-10-12 22:57:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143360000. Throughput: 0: 1646.7, 1: 1637.7. Samples: 35844720. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-12 22:57:41,444][43579] Avg episode reward: [(0, '277.780'), (1, '288.540')] [2023-10-12 22:57:43,846][44958] Updated weights for policy 0, policy_version 69830 (0.0007) [2023-10-12 22:57:44,208][44958] Updated weights for policy 0, policy_version 69840 (0.0007) [2023-10-12 22:57:44,215][44959] Updated weights for policy 1, policy_version 70180 (0.0009) [2023-10-12 22:57:44,581][44959] Updated weights for policy 1, policy_version 70190 (0.0007) [2023-10-12 22:57:44,587][44958] Updated weights for policy 0, policy_version 69850 (0.0007) [2023-10-12 22:57:44,956][44959] Updated weights for policy 1, policy_version 70200 (0.0008) [2023-10-12 22:57:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143425536. Throughput: 0: 1641.9, 1: 1638.4. Samples: 35864510. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-12 22:57:46,444][43579] Avg episode reward: [(0, '278.190'), (1, '284.780')] [2023-10-12 22:57:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000069856_71532544.pth... [2023-10-12 22:57:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000070208_71892992.pth... [2023-10-12 22:57:46,487][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000068672_70320128.pth [2023-10-12 22:57:46,495][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000068320_69959680.pth [2023-10-12 22:57:48,848][44958] Updated weights for policy 0, policy_version 69860 (0.0008) [2023-10-12 22:57:49,011][44959] Updated weights for policy 1, policy_version 70210 (0.0009) [2023-10-12 22:57:49,222][44958] Updated weights for policy 0, policy_version 69870 (0.0008) [2023-10-12 22:57:49,376][44959] Updated weights for policy 1, policy_version 70220 (0.0008) [2023-10-12 22:57:49,583][44958] Updated weights for policy 0, policy_version 69880 (0.0008) [2023-10-12 22:57:49,739][44959] Updated weights for policy 1, policy_version 70230 (0.0007) [2023-10-12 22:57:50,100][44959] Updated weights for policy 1, policy_version 70240 (0.0009) [2023-10-12 22:57:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143491072. Throughput: 0: 1634.3, 1: 1638.8. Samples: 35875196. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-12 22:57:51,444][43579] Avg episode reward: [(0, '273.540'), (1, '287.810')] [2023-10-12 22:57:53,835][44958] Updated weights for policy 0, policy_version 69890 (0.0008) [2023-10-12 22:57:54,204][44958] Updated weights for policy 0, policy_version 69900 (0.0009) [2023-10-12 22:57:54,366][44959] Updated weights for policy 1, policy_version 70250 (0.0009) [2023-10-12 22:57:54,572][44958] Updated weights for policy 0, policy_version 69910 (0.0008) [2023-10-12 22:57:54,728][44959] Updated weights for policy 1, policy_version 70260 (0.0008) [2023-10-12 22:57:54,943][44958] Updated weights for policy 0, policy_version 69920 (0.0009) [2023-10-12 22:57:55,091][44959] Updated weights for policy 1, policy_version 70270 (0.0008) [2023-10-12 22:57:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143556608. Throughput: 0: 1640.4, 1: 1643.6. Samples: 35893786. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-12 22:57:56,443][43579] Avg episode reward: [(0, '269.660'), (1, '286.850')] [2023-10-12 22:57:59,275][44958] Updated weights for policy 0, policy_version 69930 (0.0009) [2023-10-12 22:57:59,328][44959] Updated weights for policy 1, policy_version 70280 (0.0008) [2023-10-12 22:57:59,656][44958] Updated weights for policy 0, policy_version 69940 (0.0009) [2023-10-12 22:57:59,700][44959] Updated weights for policy 1, policy_version 70290 (0.0007) [2023-10-12 22:58:00,022][44958] Updated weights for policy 0, policy_version 69950 (0.0008) [2023-10-12 22:58:00,059][44959] Updated weights for policy 1, policy_version 70300 (0.0007) [2023-10-12 22:58:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 143622144. Throughput: 0: 1635.2, 1: 1647.2. Samples: 35913512. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-12 22:58:01,444][43579] Avg episode reward: [(0, '269.020'), (1, '289.450')] [2023-10-12 22:58:04,185][44958] Updated weights for policy 0, policy_version 69960 (0.0009) [2023-10-12 22:58:04,326][44959] Updated weights for policy 1, policy_version 70310 (0.0008) [2023-10-12 22:58:04,561][44958] Updated weights for policy 0, policy_version 69970 (0.0008) [2023-10-12 22:58:04,720][44959] Updated weights for policy 1, policy_version 70320 (0.0009) [2023-10-12 22:58:04,922][44958] Updated weights for policy 0, policy_version 69980 (0.0008) [2023-10-12 22:58:05,076][44959] Updated weights for policy 1, policy_version 70330 (0.0008) [2023-10-12 22:58:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143687680. Throughput: 0: 1639.2, 1: 1645.8. Samples: 35924498. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-12 22:58:06,443][43579] Avg episode reward: [(0, '267.830'), (1, '287.920')] [2023-10-12 22:58:09,085][44958] Updated weights for policy 0, policy_version 69990 (0.0008) [2023-10-12 22:58:09,188][44959] Updated weights for policy 1, policy_version 70340 (0.0007) [2023-10-12 22:58:09,462][44958] Updated weights for policy 0, policy_version 70000 (0.0009) [2023-10-12 22:58:09,558][44959] Updated weights for policy 1, policy_version 70350 (0.0007) [2023-10-12 22:58:09,843][44958] Updated weights for policy 0, policy_version 70010 (0.0008) [2023-10-12 22:58:09,922][44959] Updated weights for policy 1, policy_version 70360 (0.0007) [2023-10-12 22:58:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143753216. Throughput: 0: 1635.3, 1: 1641.1. Samples: 35942690. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-12 22:58:11,443][43579] Avg episode reward: [(0, '268.120'), (1, '290.530')] [2023-10-12 22:58:13,980][44958] Updated weights for policy 0, policy_version 70020 (0.0008) [2023-10-12 22:58:14,288][44959] Updated weights for policy 1, policy_version 70370 (0.0008) [2023-10-12 22:58:14,349][44958] Updated weights for policy 0, policy_version 70030 (0.0007) [2023-10-12 22:58:14,663][44959] Updated weights for policy 1, policy_version 70380 (0.0007) [2023-10-12 22:58:14,714][44958] Updated weights for policy 0, policy_version 70040 (0.0007) [2023-10-12 22:58:15,034][44959] Updated weights for policy 1, policy_version 70390 (0.0008) [2023-10-12 22:58:15,390][44959] Updated weights for policy 1, policy_version 70400 (0.0008) [2023-10-12 22:58:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143818752. Throughput: 0: 1628.4, 1: 1642.5. Samples: 35962550. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-12 22:58:16,443][43579] Avg episode reward: [(0, '266.980'), (1, '290.760')] [2023-10-12 22:58:18,838][44958] Updated weights for policy 0, policy_version 70050 (0.0007) [2023-10-12 22:58:19,251][44958] Updated weights for policy 0, policy_version 70060 (0.0007) [2023-10-12 22:58:19,358][44959] Updated weights for policy 1, policy_version 70410 (0.0009) [2023-10-12 22:58:19,620][44958] Updated weights for policy 0, policy_version 70070 (0.0008) [2023-10-12 22:58:19,734][44959] Updated weights for policy 1, policy_version 70420 (0.0009) [2023-10-12 22:58:19,996][44958] Updated weights for policy 0, policy_version 70080 (0.0009) [2023-10-12 22:58:20,098][44959] Updated weights for policy 1, policy_version 70430 (0.0008) [2023-10-12 22:58:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 143884288. Throughput: 0: 1631.1, 1: 1642.4. Samples: 35973520. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-12 22:58:21,443][43579] Avg episode reward: [(0, '272.100'), (1, '284.730')] [2023-10-12 22:58:24,048][44958] Updated weights for policy 0, policy_version 70090 (0.0010) [2023-10-12 22:58:24,307][44959] Updated weights for policy 1, policy_version 70440 (0.0008) [2023-10-12 22:58:24,417][44958] Updated weights for policy 0, policy_version 70100 (0.0008) [2023-10-12 22:58:24,671][44959] Updated weights for policy 1, policy_version 70450 (0.0008) [2023-10-12 22:58:24,776][44958] Updated weights for policy 0, policy_version 70110 (0.0007) [2023-10-12 22:58:25,038][44959] Updated weights for policy 1, policy_version 70460 (0.0007) [2023-10-12 22:58:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 143949824. Throughput: 0: 1624.2, 1: 1640.4. Samples: 35991626. Policy #0 lag: (min: 9.0, avg: 10.1, max: 31.0) [2023-10-12 22:58:26,443][43579] Avg episode reward: [(0, '278.780'), (1, '276.390')] [2023-10-12 22:58:29,211][44959] Updated weights for policy 1, policy_version 70470 (0.0009) [2023-10-12 22:58:29,242][44958] Updated weights for policy 0, policy_version 70120 (0.0007) [2023-10-12 22:58:29,577][44959] Updated weights for policy 1, policy_version 70480 (0.0007) [2023-10-12 22:58:29,605][44958] Updated weights for policy 0, policy_version 70130 (0.0008) [2023-10-12 22:58:29,944][44959] Updated weights for policy 1, policy_version 70490 (0.0007) [2023-10-12 22:58:29,978][44958] Updated weights for policy 0, policy_version 70140 (0.0008) [2023-10-12 22:58:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144015360. Throughput: 0: 1625.6, 1: 1645.5. Samples: 36011710. Policy #0 lag: (min: 9.0, avg: 10.1, max: 31.0) [2023-10-12 22:58:31,444][43579] Avg episode reward: [(0, '280.370'), (1, '274.890')] [2023-10-12 22:58:34,014][44958] Updated weights for policy 0, policy_version 70150 (0.0009) [2023-10-12 22:58:34,019][44959] Updated weights for policy 1, policy_version 70500 (0.0007) [2023-10-12 22:58:34,380][44958] Updated weights for policy 0, policy_version 70160 (0.0008) [2023-10-12 22:58:34,380][44959] Updated weights for policy 1, policy_version 70510 (0.0007) [2023-10-12 22:58:34,744][44959] Updated weights for policy 1, policy_version 70520 (0.0008) [2023-10-12 22:58:34,748][44958] Updated weights for policy 0, policy_version 70170 (0.0007) [2023-10-12 22:58:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 144080896. Throughput: 0: 1636.5, 1: 1636.3. Samples: 36022470. Policy #0 lag: (min: 9.0, avg: 10.1, max: 31.0) [2023-10-12 22:58:36,443][43579] Avg episode reward: [(0, '279.800'), (1, '264.410')] [2023-10-12 22:58:38,878][44959] Updated weights for policy 1, policy_version 70530 (0.0010) [2023-10-12 22:58:39,076][44958] Updated weights for policy 0, policy_version 70180 (0.0008) [2023-10-12 22:58:39,243][44959] Updated weights for policy 1, policy_version 70540 (0.0009) [2023-10-12 22:58:39,452][44958] Updated weights for policy 0, policy_version 70190 (0.0008) [2023-10-12 22:58:39,614][44959] Updated weights for policy 1, policy_version 70550 (0.0009) [2023-10-12 22:58:39,835][44958] Updated weights for policy 0, policy_version 70200 (0.0008) [2023-10-12 22:58:39,991][44959] Updated weights for policy 1, policy_version 70560 (0.0008) [2023-10-12 22:58:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144146432. Throughput: 0: 1632.6, 1: 1635.9. Samples: 36040870. Policy #0 lag: (min: 9.0, avg: 10.1, max: 31.0) [2023-10-12 22:58:41,443][43579] Avg episode reward: [(0, '282.370'), (1, '261.070')] [2023-10-12 22:58:44,074][44959] Updated weights for policy 1, policy_version 70570 (0.0009) [2023-10-12 22:58:44,105][44958] Updated weights for policy 0, policy_version 70210 (0.0009) [2023-10-12 22:58:44,439][44959] Updated weights for policy 1, policy_version 70580 (0.0009) [2023-10-12 22:58:44,477][44958] Updated weights for policy 0, policy_version 70220 (0.0009) [2023-10-12 22:58:44,817][44959] Updated weights for policy 1, policy_version 70590 (0.0009) [2023-10-12 22:58:44,839][44958] Updated weights for policy 0, policy_version 70230 (0.0007) [2023-10-12 22:58:45,221][44958] Updated weights for policy 0, policy_version 70240 (0.0008) [2023-10-12 22:58:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144211968. Throughput: 0: 1630.8, 1: 1644.0. Samples: 36060874. Policy #0 lag: (min: 9.0, avg: 10.1, max: 31.0) [2023-10-12 22:58:46,443][43579] Avg episode reward: [(0, '283.230'), (1, '259.100')] [2023-10-12 22:58:49,177][44959] Updated weights for policy 1, policy_version 70600 (0.0007) [2023-10-12 22:58:49,363][44958] Updated weights for policy 0, policy_version 70250 (0.0008) [2023-10-12 22:58:49,547][44959] Updated weights for policy 1, policy_version 70610 (0.0008) [2023-10-12 22:58:49,739][44958] Updated weights for policy 0, policy_version 70260 (0.0009) [2023-10-12 22:58:49,918][44959] Updated weights for policy 1, policy_version 70620 (0.0009) [2023-10-12 22:58:50,103][44958] Updated weights for policy 0, policy_version 70270 (0.0009) [2023-10-12 22:58:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144277504. Throughput: 0: 1632.8, 1: 1636.1. Samples: 36071600. Policy #0 lag: (min: 9.0, avg: 10.1, max: 31.0) [2023-10-12 22:58:51,443][43579] Avg episode reward: [(0, '283.630'), (1, '264.970')] [2023-10-12 22:58:54,097][44959] Updated weights for policy 1, policy_version 70630 (0.0008) [2023-10-12 22:58:54,184][44958] Updated weights for policy 0, policy_version 70280 (0.0008) [2023-10-12 22:58:54,463][44959] Updated weights for policy 1, policy_version 70640 (0.0009) [2023-10-12 22:58:54,549][44958] Updated weights for policy 0, policy_version 70290 (0.0009) [2023-10-12 22:58:54,830][44959] Updated weights for policy 1, policy_version 70650 (0.0007) [2023-10-12 22:58:54,919][44958] Updated weights for policy 0, policy_version 70300 (0.0007) [2023-10-12 22:58:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144343040. Throughput: 0: 1631.3, 1: 1639.7. Samples: 36089884. Policy #0 lag: (min: 9.0, avg: 10.1, max: 31.0) [2023-10-12 22:58:56,444][43579] Avg episode reward: [(0, '282.110'), (1, '270.450')] [2023-10-12 22:58:59,031][44959] Updated weights for policy 1, policy_version 70660 (0.0008) [2023-10-12 22:58:59,214][44958] Updated weights for policy 0, policy_version 70310 (0.0009) [2023-10-12 22:58:59,393][44959] Updated weights for policy 1, policy_version 70670 (0.0008) [2023-10-12 22:58:59,581][44958] Updated weights for policy 0, policy_version 70320 (0.0009) [2023-10-12 22:58:59,768][44959] Updated weights for policy 1, policy_version 70680 (0.0010) [2023-10-12 22:58:59,957][44958] Updated weights for policy 0, policy_version 70330 (0.0008) [2023-10-12 22:59:01,442][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144408576. Throughput: 0: 1631.6, 1: 1638.6. Samples: 36109712. Policy #0 lag: (min: 9.0, avg: 10.1, max: 31.0) [2023-10-12 22:59:01,443][43579] Avg episode reward: [(0, '277.890'), (1, '276.120')] [2023-10-12 22:59:04,096][44959] Updated weights for policy 1, policy_version 70690 (0.0008) [2023-10-12 22:59:04,207][44958] Updated weights for policy 0, policy_version 70340 (0.0009) [2023-10-12 22:59:04,465][44959] Updated weights for policy 1, policy_version 70700 (0.0009) [2023-10-12 22:59:04,585][44958] Updated weights for policy 0, policy_version 70350 (0.0007) [2023-10-12 22:59:04,839][44959] Updated weights for policy 1, policy_version 70710 (0.0008) [2023-10-12 22:59:04,957][44958] Updated weights for policy 0, policy_version 70360 (0.0010) [2023-10-12 22:59:05,197][44959] Updated weights for policy 1, policy_version 70720 (0.0008) [2023-10-12 22:59:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144474112. Throughput: 0: 1640.7, 1: 1634.0. Samples: 36120878. Policy #0 lag: (min: 9.0, avg: 10.1, max: 31.0) [2023-10-12 22:59:06,443][43579] Avg episode reward: [(0, '277.080'), (1, '276.290')] [2023-10-12 22:59:09,020][44958] Updated weights for policy 0, policy_version 70370 (0.0008) [2023-10-12 22:59:09,390][44958] Updated weights for policy 0, policy_version 70380 (0.0009) [2023-10-12 22:59:09,548][44959] Updated weights for policy 1, policy_version 70730 (0.0009) [2023-10-12 22:59:09,762][44958] Updated weights for policy 0, policy_version 70390 (0.0007) [2023-10-12 22:59:09,917][44959] Updated weights for policy 1, policy_version 70740 (0.0010) [2023-10-12 22:59:10,126][44958] Updated weights for policy 0, policy_version 70400 (0.0008) [2023-10-12 22:59:10,282][44959] Updated weights for policy 1, policy_version 70750 (0.0007) [2023-10-12 22:59:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144539648. Throughput: 0: 1636.0, 1: 1640.5. Samples: 36139070. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 22:59:11,443][43579] Avg episode reward: [(0, '277.060'), (1, '273.400')] [2023-10-12 22:59:14,272][44958] Updated weights for policy 0, policy_version 70410 (0.0007) [2023-10-12 22:59:14,579][44959] Updated weights for policy 1, policy_version 70760 (0.0010) [2023-10-12 22:59:14,642][44958] Updated weights for policy 0, policy_version 70420 (0.0008) [2023-10-12 22:59:14,946][44959] Updated weights for policy 1, policy_version 70770 (0.0008) [2023-10-12 22:59:15,017][44958] Updated weights for policy 0, policy_version 70430 (0.0008) [2023-10-12 22:59:15,312][44959] Updated weights for policy 1, policy_version 70780 (0.0009) [2023-10-12 22:59:16,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144605184. Throughput: 0: 1636.9, 1: 1627.6. Samples: 36158612. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 22:59:16,444][43579] Avg episode reward: [(0, '279.880'), (1, '272.130')] [2023-10-12 22:59:19,227][44958] Updated weights for policy 0, policy_version 70440 (0.0009) [2023-10-12 22:59:19,233][44959] Updated weights for policy 1, policy_version 70790 (0.0008) [2023-10-12 22:59:19,599][44958] Updated weights for policy 0, policy_version 70450 (0.0008) [2023-10-12 22:59:19,601][44959] Updated weights for policy 1, policy_version 70800 (0.0007) [2023-10-12 22:59:19,966][44959] Updated weights for policy 1, policy_version 70810 (0.0009) [2023-10-12 22:59:19,971][44958] Updated weights for policy 0, policy_version 70460 (0.0008) [2023-10-12 22:59:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144670720. Throughput: 0: 1639.6, 1: 1636.9. Samples: 36169910. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 22:59:21,443][43579] Avg episode reward: [(0, '268.470'), (1, '266.050')] [2023-10-12 22:59:24,095][44958] Updated weights for policy 0, policy_version 70470 (0.0009) [2023-10-12 22:59:24,324][44959] Updated weights for policy 1, policy_version 70820 (0.0009) [2023-10-12 22:59:24,469][44958] Updated weights for policy 0, policy_version 70480 (0.0009) [2023-10-12 22:59:24,700][44959] Updated weights for policy 1, policy_version 70830 (0.0009) [2023-10-12 22:59:24,839][44958] Updated weights for policy 0, policy_version 70490 (0.0007) [2023-10-12 22:59:25,074][44959] Updated weights for policy 1, policy_version 70840 (0.0008) [2023-10-12 22:59:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144736256. Throughput: 0: 1636.4, 1: 1638.0. Samples: 36188222. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 22:59:26,443][43579] Avg episode reward: [(0, '266.610'), (1, '267.540')] [2023-10-12 22:59:29,013][44958] Updated weights for policy 0, policy_version 70500 (0.0009) [2023-10-12 22:59:29,344][44959] Updated weights for policy 1, policy_version 70850 (0.0008) [2023-10-12 22:59:29,376][44958] Updated weights for policy 0, policy_version 70510 (0.0009) [2023-10-12 22:59:29,709][44959] Updated weights for policy 1, policy_version 70860 (0.0008) [2023-10-12 22:59:29,760][44958] Updated weights for policy 0, policy_version 70520 (0.0008) [2023-10-12 22:59:30,082][44959] Updated weights for policy 1, policy_version 70870 (0.0008) [2023-10-12 22:59:30,446][44959] Updated weights for policy 1, policy_version 70880 (0.0009) [2023-10-12 22:59:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144801792. Throughput: 0: 1640.9, 1: 1626.3. Samples: 36207898. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 22:59:31,443][43579] Avg episode reward: [(0, '273.010'), (1, '269.220')] [2023-10-12 22:59:33,963][44958] Updated weights for policy 0, policy_version 70530 (0.0009) [2023-10-12 22:59:34,337][44958] Updated weights for policy 0, policy_version 70540 (0.0008) [2023-10-12 22:59:34,705][44958] Updated weights for policy 0, policy_version 70550 (0.0007) [2023-10-12 22:59:34,783][44959] Updated weights for policy 1, policy_version 70890 (0.0009) [2023-10-12 22:59:35,072][44958] Updated weights for policy 0, policy_version 70560 (0.0007) [2023-10-12 22:59:35,146][44959] Updated weights for policy 1, policy_version 70900 (0.0008) [2023-10-12 22:59:35,515][44959] Updated weights for policy 1, policy_version 70910 (0.0008) [2023-10-12 22:59:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144867328. Throughput: 0: 1639.0, 1: 1635.7. Samples: 36218962. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 22:59:36,443][43579] Avg episode reward: [(0, '273.600'), (1, '278.920')] [2023-10-12 22:59:39,280][44958] Updated weights for policy 0, policy_version 70570 (0.0009) [2023-10-12 22:59:39,657][44958] Updated weights for policy 0, policy_version 70580 (0.0008) [2023-10-12 22:59:39,764][44959] Updated weights for policy 1, policy_version 70920 (0.0008) [2023-10-12 22:59:40,027][44958] Updated weights for policy 0, policy_version 70590 (0.0009) [2023-10-12 22:59:40,126][44959] Updated weights for policy 1, policy_version 70930 (0.0008) [2023-10-12 22:59:40,508][44959] Updated weights for policy 1, policy_version 70940 (0.0008) [2023-10-12 22:59:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144932864. Throughput: 0: 1637.3, 1: 1634.4. Samples: 36237112. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 22:59:41,443][43579] Avg episode reward: [(0, '271.400'), (1, '279.190')] [2023-10-12 22:59:44,100][44958] Updated weights for policy 0, policy_version 70600 (0.0007) [2023-10-12 22:59:44,475][44958] Updated weights for policy 0, policy_version 70610 (0.0008) [2023-10-12 22:59:44,520][44959] Updated weights for policy 1, policy_version 70950 (0.0008) [2023-10-12 22:59:44,861][44958] Updated weights for policy 0, policy_version 70620 (0.0008) [2023-10-12 22:59:44,897][44959] Updated weights for policy 1, policy_version 70960 (0.0008) [2023-10-12 22:59:45,267][44959] Updated weights for policy 1, policy_version 70970 (0.0009) [2023-10-12 22:59:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 144998400. Throughput: 0: 1646.6, 1: 1625.2. Samples: 36256946. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 22:59:46,444][43579] Avg episode reward: [(0, '271.110'), (1, '284.720')] [2023-10-12 22:59:46,456][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000070624_72318976.pth... [2023-10-12 22:59:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000070976_72679424.pth... [2023-10-12 22:59:46,494][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000069440_71106560.pth [2023-10-12 22:59:46,501][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000069088_70746112.pth [2023-10-12 22:59:49,074][44958] Updated weights for policy 0, policy_version 70630 (0.0009) [2023-10-12 22:59:49,453][44958] Updated weights for policy 0, policy_version 70640 (0.0010) [2023-10-12 22:59:49,455][44959] Updated weights for policy 1, policy_version 70980 (0.0009) [2023-10-12 22:59:49,816][44959] Updated weights for policy 1, policy_version 70990 (0.0010) [2023-10-12 22:59:49,819][44958] Updated weights for policy 0, policy_version 70650 (0.0009) [2023-10-12 22:59:50,190][44959] Updated weights for policy 1, policy_version 71000 (0.0009) [2023-10-12 22:59:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145063936. Throughput: 0: 1635.0, 1: 1626.2. Samples: 36267634. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-12 22:59:51,443][43579] Avg episode reward: [(0, '278.400'), (1, '285.920')] [2023-10-12 22:59:53,971][44958] Updated weights for policy 0, policy_version 70660 (0.0008) [2023-10-12 22:59:54,348][44958] Updated weights for policy 0, policy_version 70670 (0.0008) [2023-10-12 22:59:54,373][44959] Updated weights for policy 1, policy_version 71010 (0.0009) [2023-10-12 22:59:54,712][44958] Updated weights for policy 0, policy_version 70680 (0.0007) [2023-10-12 22:59:54,749][44959] Updated weights for policy 1, policy_version 71020 (0.0009) [2023-10-12 22:59:55,121][44959] Updated weights for policy 1, policy_version 71030 (0.0008) [2023-10-12 22:59:55,488][44959] Updated weights for policy 1, policy_version 71040 (0.0007) [2023-10-12 22:59:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145129472. Throughput: 0: 1642.4, 1: 1631.5. Samples: 36286394. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 22:59:56,443][43579] Avg episode reward: [(0, '276.030'), (1, '283.160')] [2023-10-12 22:59:58,977][44958] Updated weights for policy 0, policy_version 70690 (0.0008) [2023-10-12 22:59:59,350][44958] Updated weights for policy 0, policy_version 70700 (0.0010) [2023-10-12 22:59:59,691][44959] Updated weights for policy 1, policy_version 71050 (0.0008) [2023-10-12 22:59:59,717][44958] Updated weights for policy 0, policy_version 70710 (0.0008) [2023-10-12 23:00:00,058][44959] Updated weights for policy 1, policy_version 71060 (0.0008) [2023-10-12 23:00:00,091][44958] Updated weights for policy 0, policy_version 70720 (0.0009) [2023-10-12 23:00:00,421][44959] Updated weights for policy 1, policy_version 71070 (0.0007) [2023-10-12 23:00:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145195008. Throughput: 0: 1638.0, 1: 1635.9. Samples: 36305936. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 23:00:01,444][43579] Avg episode reward: [(0, '269.760'), (1, '284.380')] [2023-10-12 23:00:04,398][44958] Updated weights for policy 0, policy_version 70730 (0.0008) [2023-10-12 23:00:04,515][44959] Updated weights for policy 1, policy_version 71080 (0.0008) [2023-10-12 23:00:04,769][44958] Updated weights for policy 0, policy_version 70740 (0.0007) [2023-10-12 23:00:04,886][44959] Updated weights for policy 1, policy_version 71090 (0.0009) [2023-10-12 23:00:05,145][44958] Updated weights for policy 0, policy_version 70750 (0.0007) [2023-10-12 23:00:05,249][44959] Updated weights for policy 1, policy_version 71100 (0.0008) [2023-10-12 23:00:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145260544. Throughput: 0: 1637.2, 1: 1632.6. Samples: 36317050. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 23:00:06,443][43579] Avg episode reward: [(0, '268.890'), (1, '284.500')] [2023-10-12 23:00:09,339][44958] Updated weights for policy 0, policy_version 70760 (0.0007) [2023-10-12 23:00:09,505][44959] Updated weights for policy 1, policy_version 71110 (0.0008) [2023-10-12 23:00:09,716][44958] Updated weights for policy 0, policy_version 70770 (0.0007) [2023-10-12 23:00:09,876][44959] Updated weights for policy 1, policy_version 71120 (0.0007) [2023-10-12 23:00:10,072][44958] Updated weights for policy 0, policy_version 70780 (0.0008) [2023-10-12 23:00:10,241][44959] Updated weights for policy 1, policy_version 71130 (0.0007) [2023-10-12 23:00:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145326080. Throughput: 0: 1633.7, 1: 1633.0. Samples: 36335226. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 23:00:11,443][43579] Avg episode reward: [(0, '270.300'), (1, '277.730')] [2023-10-12 23:00:14,316][44958] Updated weights for policy 0, policy_version 70790 (0.0008) [2023-10-12 23:00:14,323][44959] Updated weights for policy 1, policy_version 71140 (0.0008) [2023-10-12 23:00:14,684][44958] Updated weights for policy 0, policy_version 70800 (0.0009) [2023-10-12 23:00:14,706][44959] Updated weights for policy 1, policy_version 71150 (0.0008) [2023-10-12 23:00:15,052][44958] Updated weights for policy 0, policy_version 70810 (0.0007) [2023-10-12 23:00:15,071][44959] Updated weights for policy 1, policy_version 71160 (0.0007) [2023-10-12 23:00:16,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145391616. Throughput: 0: 1630.5, 1: 1636.6. Samples: 36354918. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 23:00:16,444][43579] Avg episode reward: [(0, '270.630'), (1, '278.260')] [2023-10-12 23:00:19,060][44958] Updated weights for policy 0, policy_version 70820 (0.0008) [2023-10-12 23:00:19,285][44959] Updated weights for policy 1, policy_version 71170 (0.0009) [2023-10-12 23:00:19,427][44958] Updated weights for policy 0, policy_version 70830 (0.0008) [2023-10-12 23:00:19,704][44959] Updated weights for policy 1, policy_version 71180 (0.0009) [2023-10-12 23:00:19,806][44958] Updated weights for policy 0, policy_version 70840 (0.0008) [2023-10-12 23:00:20,076][44959] Updated weights for policy 1, policy_version 71190 (0.0008) [2023-10-12 23:00:20,451][44959] Updated weights for policy 1, policy_version 71200 (0.0008) [2023-10-12 23:00:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145457152. Throughput: 0: 1631.2, 1: 1632.7. Samples: 36365838. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 23:00:21,443][43579] Avg episode reward: [(0, '270.350'), (1, '283.680')] [2023-10-12 23:00:24,047][44958] Updated weights for policy 0, policy_version 70850 (0.0009) [2023-10-12 23:00:24,416][44958] Updated weights for policy 0, policy_version 70860 (0.0008) [2023-10-12 23:00:24,522][44959] Updated weights for policy 1, policy_version 71210 (0.0009) [2023-10-12 23:00:24,789][44958] Updated weights for policy 0, policy_version 70870 (0.0009) [2023-10-12 23:00:24,895][44959] Updated weights for policy 1, policy_version 71220 (0.0008) [2023-10-12 23:00:25,169][44958] Updated weights for policy 0, policy_version 70880 (0.0007) [2023-10-12 23:00:25,256][44959] Updated weights for policy 1, policy_version 71230 (0.0009) [2023-10-12 23:00:26,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145522688. Throughput: 0: 1634.5, 1: 1633.1. Samples: 36384154. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 23:00:26,444][43579] Avg episode reward: [(0, '273.540'), (1, '283.690')] [2023-10-12 23:00:29,450][44958] Updated weights for policy 0, policy_version 70890 (0.0010) [2023-10-12 23:00:29,531][44959] Updated weights for policy 1, policy_version 71240 (0.0010) [2023-10-12 23:00:29,822][44958] Updated weights for policy 0, policy_version 70900 (0.0009) [2023-10-12 23:00:29,901][44959] Updated weights for policy 1, policy_version 71250 (0.0008) [2023-10-12 23:00:30,180][44958] Updated weights for policy 0, policy_version 70910 (0.0010) [2023-10-12 23:00:30,275][44959] Updated weights for policy 1, policy_version 71260 (0.0008) [2023-10-12 23:00:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145588224. Throughput: 0: 1621.5, 1: 1636.5. Samples: 36403556. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 23:00:31,444][43579] Avg episode reward: [(0, '277.360'), (1, '284.160')] [2023-10-12 23:00:34,356][44958] Updated weights for policy 0, policy_version 70920 (0.0009) [2023-10-12 23:00:34,500][44959] Updated weights for policy 1, policy_version 71270 (0.0008) [2023-10-12 23:00:34,720][44958] Updated weights for policy 0, policy_version 70930 (0.0007) [2023-10-12 23:00:34,868][44959] Updated weights for policy 1, policy_version 71280 (0.0007) [2023-10-12 23:00:35,093][44958] Updated weights for policy 0, policy_version 70940 (0.0008) [2023-10-12 23:00:35,231][44959] Updated weights for policy 1, policy_version 71290 (0.0008) [2023-10-12 23:00:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145653760. Throughput: 0: 1629.0, 1: 1635.5. Samples: 36414538. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-12 23:00:36,444][43579] Avg episode reward: [(0, '281.270'), (1, '277.200')] [2023-10-12 23:00:39,275][44958] Updated weights for policy 0, policy_version 70950 (0.0008) [2023-10-12 23:00:39,645][44958] Updated weights for policy 0, policy_version 70960 (0.0008) [2023-10-12 23:00:39,718][44959] Updated weights for policy 1, policy_version 71300 (0.0009) [2023-10-12 23:00:40,016][44958] Updated weights for policy 0, policy_version 70970 (0.0007) [2023-10-12 23:00:40,085][44959] Updated weights for policy 1, policy_version 71310 (0.0008) [2023-10-12 23:00:40,455][44959] Updated weights for policy 1, policy_version 71320 (0.0008) [2023-10-12 23:00:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145719296. Throughput: 0: 1625.6, 1: 1632.5. Samples: 36433008. Policy #0 lag: (min: 7.0, avg: 9.1, max: 39.0) [2023-10-12 23:00:41,443][43579] Avg episode reward: [(0, '281.140'), (1, '278.190')] [2023-10-12 23:00:44,225][44958] Updated weights for policy 0, policy_version 70980 (0.0008) [2023-10-12 23:00:44,566][44959] Updated weights for policy 1, policy_version 71330 (0.0008) [2023-10-12 23:00:44,604][44958] Updated weights for policy 0, policy_version 70990 (0.0008) [2023-10-12 23:00:44,932][44959] Updated weights for policy 1, policy_version 71340 (0.0007) [2023-10-12 23:00:44,968][44958] Updated weights for policy 0, policy_version 71000 (0.0009) [2023-10-12 23:00:45,304][44959] Updated weights for policy 1, policy_version 71350 (0.0009) [2023-10-12 23:00:45,666][44959] Updated weights for policy 1, policy_version 71360 (0.0009) [2023-10-12 23:00:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145784832. Throughput: 0: 1622.6, 1: 1629.2. Samples: 36452264. Policy #0 lag: (min: 7.0, avg: 9.1, max: 39.0) [2023-10-12 23:00:46,444][43579] Avg episode reward: [(0, '284.550'), (1, '279.370')] [2023-10-12 23:00:49,199][44958] Updated weights for policy 0, policy_version 71010 (0.0008) [2023-10-12 23:00:49,576][44958] Updated weights for policy 0, policy_version 71020 (0.0008) [2023-10-12 23:00:49,842][44959] Updated weights for policy 1, policy_version 71370 (0.0009) [2023-10-12 23:00:49,950][44958] Updated weights for policy 0, policy_version 71030 (0.0007) [2023-10-12 23:00:50,203][44959] Updated weights for policy 1, policy_version 71380 (0.0007) [2023-10-12 23:00:50,317][44958] Updated weights for policy 0, policy_version 71040 (0.0009) [2023-10-12 23:00:50,581][44959] Updated weights for policy 1, policy_version 71390 (0.0009) [2023-10-12 23:00:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145850368. Throughput: 0: 1628.5, 1: 1630.3. Samples: 36463696. Policy #0 lag: (min: 7.0, avg: 9.1, max: 39.0) [2023-10-12 23:00:51,443][43579] Avg episode reward: [(0, '286.440'), (1, '282.560')] [2023-10-12 23:00:54,539][44958] Updated weights for policy 0, policy_version 71050 (0.0008) [2023-10-12 23:00:54,744][44959] Updated weights for policy 1, policy_version 71400 (0.0008) [2023-10-12 23:00:54,907][44958] Updated weights for policy 0, policy_version 71060 (0.0008) [2023-10-12 23:00:55,097][44959] Updated weights for policy 1, policy_version 71410 (0.0008) [2023-10-12 23:00:55,277][44958] Updated weights for policy 0, policy_version 71070 (0.0008) [2023-10-12 23:00:55,472][44959] Updated weights for policy 1, policy_version 71420 (0.0008) [2023-10-12 23:00:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145915904. Throughput: 0: 1634.2, 1: 1636.8. Samples: 36482420. Policy #0 lag: (min: 7.0, avg: 9.1, max: 39.0) [2023-10-12 23:00:56,443][43579] Avg episode reward: [(0, '282.710'), (1, '282.520')] [2023-10-12 23:00:59,509][44959] Updated weights for policy 1, policy_version 71430 (0.0008) [2023-10-12 23:00:59,513][44958] Updated weights for policy 0, policy_version 71080 (0.0008) [2023-10-12 23:00:59,886][44959] Updated weights for policy 1, policy_version 71440 (0.0007) [2023-10-12 23:00:59,886][44958] Updated weights for policy 0, policy_version 71090 (0.0009) [2023-10-12 23:01:00,251][44959] Updated weights for policy 1, policy_version 71450 (0.0008) [2023-10-12 23:01:00,256][44958] Updated weights for policy 0, policy_version 71100 (0.0009) [2023-10-12 23:01:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 145981440. Throughput: 0: 1633.6, 1: 1635.8. Samples: 36502038. Policy #0 lag: (min: 7.0, avg: 9.1, max: 39.0) [2023-10-12 23:01:01,443][43579] Avg episode reward: [(0, '285.760'), (1, '281.910')] [2023-10-12 23:01:04,336][44958] Updated weights for policy 0, policy_version 71110 (0.0009) [2023-10-12 23:01:04,602][44959] Updated weights for policy 1, policy_version 71460 (0.0008) [2023-10-12 23:01:04,705][44958] Updated weights for policy 0, policy_version 71120 (0.0007) [2023-10-12 23:01:04,994][44959] Updated weights for policy 1, policy_version 71470 (0.0008) [2023-10-12 23:01:05,082][44958] Updated weights for policy 0, policy_version 71130 (0.0007) [2023-10-12 23:01:05,362][44959] Updated weights for policy 1, policy_version 71480 (0.0007) [2023-10-12 23:01:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146046976. Throughput: 0: 1640.8, 1: 1637.8. Samples: 36513372. Policy #0 lag: (min: 7.0, avg: 9.1, max: 39.0) [2023-10-12 23:01:06,443][43579] Avg episode reward: [(0, '282.390'), (1, '285.760')] [2023-10-12 23:01:09,357][44958] Updated weights for policy 0, policy_version 71140 (0.0008) [2023-10-12 23:01:09,491][44959] Updated weights for policy 1, policy_version 71490 (0.0009) [2023-10-12 23:01:09,719][44958] Updated weights for policy 0, policy_version 71150 (0.0010) [2023-10-12 23:01:09,859][44959] Updated weights for policy 1, policy_version 71500 (0.0008) [2023-10-12 23:01:10,099][44958] Updated weights for policy 0, policy_version 71160 (0.0008) [2023-10-12 23:01:10,223][44959] Updated weights for policy 1, policy_version 71510 (0.0007) [2023-10-12 23:01:10,597][44959] Updated weights for policy 1, policy_version 71520 (0.0008) [2023-10-12 23:01:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146112512. Throughput: 0: 1639.2, 1: 1638.0. Samples: 36531628. Policy #0 lag: (min: 7.0, avg: 9.1, max: 39.0) [2023-10-12 23:01:11,443][43579] Avg episode reward: [(0, '279.550'), (1, '281.750')] [2023-10-12 23:01:14,414][44958] Updated weights for policy 0, policy_version 71170 (0.0008) [2023-10-12 23:01:14,786][44958] Updated weights for policy 0, policy_version 71180 (0.0008) [2023-10-12 23:01:14,840][44959] Updated weights for policy 1, policy_version 71530 (0.0007) [2023-10-12 23:01:15,149][44958] Updated weights for policy 0, policy_version 71190 (0.0008) [2023-10-12 23:01:15,201][44959] Updated weights for policy 1, policy_version 71540 (0.0007) [2023-10-12 23:01:15,519][44958] Updated weights for policy 0, policy_version 71200 (0.0008) [2023-10-12 23:01:15,565][44959] Updated weights for policy 1, policy_version 71550 (0.0009) [2023-10-12 23:01:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146178048. Throughput: 0: 1637.6, 1: 1633.4. Samples: 36550754. Policy #0 lag: (min: 7.0, avg: 9.1, max: 39.0) [2023-10-12 23:01:16,444][43579] Avg episode reward: [(0, '279.370'), (1, '277.610')] [2023-10-12 23:01:19,701][44958] Updated weights for policy 0, policy_version 71210 (0.0009) [2023-10-12 23:01:19,828][44959] Updated weights for policy 1, policy_version 71560 (0.0009) [2023-10-12 23:01:20,069][44958] Updated weights for policy 0, policy_version 71220 (0.0008) [2023-10-12 23:01:20,189][44959] Updated weights for policy 1, policy_version 71570 (0.0008) [2023-10-12 23:01:20,445][44958] Updated weights for policy 0, policy_version 71230 (0.0007) [2023-10-12 23:01:20,563][44959] Updated weights for policy 1, policy_version 71580 (0.0009) [2023-10-12 23:01:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146243584. Throughput: 0: 1640.9, 1: 1634.0. Samples: 36561908. Policy #0 lag: (min: 7.0, avg: 9.1, max: 39.0) [2023-10-12 23:01:21,444][43579] Avg episode reward: [(0, '278.580'), (1, '277.100')] [2023-10-12 23:01:24,352][44958] Updated weights for policy 0, policy_version 71240 (0.0007) [2023-10-12 23:01:24,718][44958] Updated weights for policy 0, policy_version 71250 (0.0009) [2023-10-12 23:01:24,775][44959] Updated weights for policy 1, policy_version 71590 (0.0008) [2023-10-12 23:01:25,083][44958] Updated weights for policy 0, policy_version 71260 (0.0007) [2023-10-12 23:01:25,143][44959] Updated weights for policy 1, policy_version 71600 (0.0008) [2023-10-12 23:01:25,512][44959] Updated weights for policy 1, policy_version 71610 (0.0009) [2023-10-12 23:01:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146309120. Throughput: 0: 1640.4, 1: 1638.7. Samples: 36580568. Policy #0 lag: (min: 6.0, avg: 8.1, max: 37.0) [2023-10-12 23:01:26,444][43579] Avg episode reward: [(0, '277.080'), (1, '278.410')] [2023-10-12 23:01:29,394][44958] Updated weights for policy 0, policy_version 71270 (0.0009) [2023-10-12 23:01:29,710][44959] Updated weights for policy 1, policy_version 71620 (0.0008) [2023-10-12 23:01:29,760][44958] Updated weights for policy 0, policy_version 71280 (0.0009) [2023-10-12 23:01:30,082][44959] Updated weights for policy 1, policy_version 71630 (0.0007) [2023-10-12 23:01:30,130][44958] Updated weights for policy 0, policy_version 71290 (0.0008) [2023-10-12 23:01:30,448][44959] Updated weights for policy 1, policy_version 71640 (0.0007) [2023-10-12 23:01:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146374656. Throughput: 0: 1643.4, 1: 1634.2. Samples: 36599756. Policy #0 lag: (min: 6.0, avg: 8.1, max: 37.0) [2023-10-12 23:01:31,444][43579] Avg episode reward: [(0, '267.310'), (1, '272.090')] [2023-10-12 23:01:34,233][44958] Updated weights for policy 0, policy_version 71300 (0.0007) [2023-10-12 23:01:34,600][44958] Updated weights for policy 0, policy_version 71310 (0.0008) [2023-10-12 23:01:34,631][44959] Updated weights for policy 1, policy_version 71650 (0.0008) [2023-10-12 23:01:34,966][44958] Updated weights for policy 0, policy_version 71320 (0.0008) [2023-10-12 23:01:34,999][44959] Updated weights for policy 1, policy_version 71660 (0.0009) [2023-10-12 23:01:35,361][44959] Updated weights for policy 1, policy_version 71670 (0.0009) [2023-10-12 23:01:35,734][44959] Updated weights for policy 1, policy_version 71680 (0.0008) [2023-10-12 23:01:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146440192. Throughput: 0: 1644.6, 1: 1632.4. Samples: 36611162. Policy #0 lag: (min: 6.0, avg: 8.1, max: 37.0) [2023-10-12 23:01:36,443][43579] Avg episode reward: [(0, '270.740'), (1, '277.710')] [2023-10-12 23:01:39,178][44958] Updated weights for policy 0, policy_version 71330 (0.0009) [2023-10-12 23:01:39,552][44958] Updated weights for policy 0, policy_version 71340 (0.0007) [2023-10-12 23:01:39,917][44958] Updated weights for policy 0, policy_version 71350 (0.0008) [2023-10-12 23:01:39,980][44959] Updated weights for policy 1, policy_version 71690 (0.0008) [2023-10-12 23:01:40,294][44958] Updated weights for policy 0, policy_version 71360 (0.0010) [2023-10-12 23:01:40,351][44959] Updated weights for policy 1, policy_version 71700 (0.0008) [2023-10-12 23:01:40,717][44959] Updated weights for policy 1, policy_version 71710 (0.0010) [2023-10-12 23:01:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146505728. Throughput: 0: 1640.4, 1: 1641.4. Samples: 36630102. Policy #0 lag: (min: 6.0, avg: 8.1, max: 37.0) [2023-10-12 23:01:41,443][43579] Avg episode reward: [(0, '269.540'), (1, '282.650')] [2023-10-12 23:01:44,447][44958] Updated weights for policy 0, policy_version 71370 (0.0009) [2023-10-12 23:01:44,824][44958] Updated weights for policy 0, policy_version 71380 (0.0008) [2023-10-12 23:01:44,840][44959] Updated weights for policy 1, policy_version 71720 (0.0007) [2023-10-12 23:01:45,198][44958] Updated weights for policy 0, policy_version 71390 (0.0009) [2023-10-12 23:01:45,212][44959] Updated weights for policy 1, policy_version 71730 (0.0009) [2023-10-12 23:01:45,577][44959] Updated weights for policy 1, policy_version 71740 (0.0008) [2023-10-12 23:01:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146571264. Throughput: 0: 1641.5, 1: 1629.5. Samples: 36649234. Policy #0 lag: (min: 6.0, avg: 8.1, max: 37.0) [2023-10-12 23:01:46,444][43579] Avg episode reward: [(0, '259.500'), (1, '282.160')] [2023-10-12 23:01:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000071392_73105408.pth... [2023-10-12 23:01:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000071744_73465856.pth... [2023-10-12 23:01:46,505][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000069856_71532544.pth [2023-10-12 23:01:46,506][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000070208_71892992.pth [2023-10-12 23:01:49,726][44958] Updated weights for policy 0, policy_version 71400 (0.0008) [2023-10-12 23:01:49,797][44959] Updated weights for policy 1, policy_version 71750 (0.0009) [2023-10-12 23:01:50,094][44958] Updated weights for policy 0, policy_version 71410 (0.0007) [2023-10-12 23:01:50,183][44959] Updated weights for policy 1, policy_version 71760 (0.0008) [2023-10-12 23:01:50,458][44958] Updated weights for policy 0, policy_version 71420 (0.0008) [2023-10-12 23:01:50,542][44959] Updated weights for policy 1, policy_version 71770 (0.0008) [2023-10-12 23:01:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146636800. Throughput: 0: 1634.1, 1: 1631.0. Samples: 36660302. Policy #0 lag: (min: 6.0, avg: 8.1, max: 37.0) [2023-10-12 23:01:51,444][43579] Avg episode reward: [(0, '256.520'), (1, '281.020')] [2023-10-12 23:01:54,379][44958] Updated weights for policy 0, policy_version 71430 (0.0008) [2023-10-12 23:01:54,678][44959] Updated weights for policy 1, policy_version 71780 (0.0007) [2023-10-12 23:01:54,746][44958] Updated weights for policy 0, policy_version 71440 (0.0007) [2023-10-12 23:01:55,042][44959] Updated weights for policy 1, policy_version 71790 (0.0007) [2023-10-12 23:01:55,108][44958] Updated weights for policy 0, policy_version 71450 (0.0008) [2023-10-12 23:01:55,408][44959] Updated weights for policy 1, policy_version 71800 (0.0007) [2023-10-12 23:01:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146702336. Throughput: 0: 1637.6, 1: 1639.2. Samples: 36679084. Policy #0 lag: (min: 6.0, avg: 8.1, max: 37.0) [2023-10-12 23:01:56,443][43579] Avg episode reward: [(0, '254.820'), (1, '280.540')] [2023-10-12 23:01:59,424][44958] Updated weights for policy 0, policy_version 71460 (0.0009) [2023-10-12 23:01:59,538][44959] Updated weights for policy 1, policy_version 71810 (0.0009) [2023-10-12 23:01:59,791][44958] Updated weights for policy 0, policy_version 71470 (0.0008) [2023-10-12 23:01:59,913][44959] Updated weights for policy 1, policy_version 71820 (0.0008) [2023-10-12 23:02:00,163][44958] Updated weights for policy 0, policy_version 71480 (0.0009) [2023-10-12 23:02:00,277][44959] Updated weights for policy 1, policy_version 71830 (0.0008) [2023-10-12 23:02:00,650][44959] Updated weights for policy 1, policy_version 71840 (0.0008) [2023-10-12 23:02:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146767872. Throughput: 0: 1642.9, 1: 1635.9. Samples: 36698298. Policy #0 lag: (min: 6.0, avg: 8.1, max: 37.0) [2023-10-12 23:02:01,443][43579] Avg episode reward: [(0, '260.670'), (1, '285.390')] [2023-10-12 23:02:04,284][44958] Updated weights for policy 0, policy_version 71490 (0.0008) [2023-10-12 23:02:04,689][44958] Updated weights for policy 0, policy_version 71500 (0.0010) [2023-10-12 23:02:04,773][44959] Updated weights for policy 1, policy_version 71850 (0.0009) [2023-10-12 23:02:05,059][44958] Updated weights for policy 0, policy_version 71510 (0.0008) [2023-10-12 23:02:05,146][44959] Updated weights for policy 1, policy_version 71860 (0.0007) [2023-10-12 23:02:05,434][44958] Updated weights for policy 0, policy_version 71520 (0.0008) [2023-10-12 23:02:05,512][44959] Updated weights for policy 1, policy_version 71870 (0.0007) [2023-10-12 23:02:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146833408. Throughput: 0: 1645.0, 1: 1640.2. Samples: 36709742. Policy #0 lag: (min: 6.0, avg: 8.1, max: 37.0) [2023-10-12 23:02:06,443][43579] Avg episode reward: [(0, '261.250'), (1, '281.380')] [2023-10-12 23:02:09,545][44958] Updated weights for policy 0, policy_version 71530 (0.0009) [2023-10-12 23:02:09,632][44959] Updated weights for policy 1, policy_version 71880 (0.0008) [2023-10-12 23:02:09,923][44958] Updated weights for policy 0, policy_version 71540 (0.0008) [2023-10-12 23:02:09,990][44959] Updated weights for policy 1, policy_version 71890 (0.0009) [2023-10-12 23:02:10,286][44958] Updated weights for policy 0, policy_version 71550 (0.0009) [2023-10-12 23:02:10,359][44959] Updated weights for policy 1, policy_version 71900 (0.0008) [2023-10-12 23:02:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146898944. Throughput: 0: 1643.7, 1: 1642.1. Samples: 36728426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:11,443][43579] Avg episode reward: [(0, '260.240'), (1, '281.130')] [2023-10-12 23:02:14,357][44958] Updated weights for policy 0, policy_version 71560 (0.0009) [2023-10-12 23:02:14,585][44959] Updated weights for policy 1, policy_version 71910 (0.0008) [2023-10-12 23:02:14,715][44958] Updated weights for policy 0, policy_version 71570 (0.0007) [2023-10-12 23:02:14,954][44959] Updated weights for policy 1, policy_version 71920 (0.0009) [2023-10-12 23:02:15,090][44958] Updated weights for policy 0, policy_version 71580 (0.0007) [2023-10-12 23:02:15,331][44959] Updated weights for policy 1, policy_version 71930 (0.0009) [2023-10-12 23:02:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 146964480. Throughput: 0: 1645.1, 1: 1640.9. Samples: 36747628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:16,443][43579] Avg episode reward: [(0, '267.830'), (1, '284.540')] [2023-10-12 23:02:19,410][44958] Updated weights for policy 0, policy_version 71590 (0.0008) [2023-10-12 23:02:19,435][44959] Updated weights for policy 1, policy_version 71940 (0.0009) [2023-10-12 23:02:19,779][44958] Updated weights for policy 0, policy_version 71600 (0.0007) [2023-10-12 23:02:19,803][44959] Updated weights for policy 1, policy_version 71950 (0.0008) [2023-10-12 23:02:20,148][44958] Updated weights for policy 0, policy_version 71610 (0.0007) [2023-10-12 23:02:20,175][44959] Updated weights for policy 1, policy_version 71960 (0.0008) [2023-10-12 23:02:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147030016. Throughput: 0: 1640.3, 1: 1644.4. Samples: 36758972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:21,443][43579] Avg episode reward: [(0, '273.920'), (1, '285.940')] [2023-10-12 23:02:24,345][44958] Updated weights for policy 0, policy_version 71620 (0.0009) [2023-10-12 23:02:24,434][44959] Updated weights for policy 1, policy_version 71970 (0.0007) [2023-10-12 23:02:24,727][44958] Updated weights for policy 0, policy_version 71630 (0.0008) [2023-10-12 23:02:24,792][44959] Updated weights for policy 1, policy_version 71980 (0.0007) [2023-10-12 23:02:25,091][44958] Updated weights for policy 0, policy_version 71640 (0.0009) [2023-10-12 23:02:25,168][44959] Updated weights for policy 1, policy_version 71990 (0.0008) [2023-10-12 23:02:25,541][44959] Updated weights for policy 1, policy_version 72000 (0.0009) [2023-10-12 23:02:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147095552. Throughput: 0: 1638.0, 1: 1634.9. Samples: 36777380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:26,444][43579] Avg episode reward: [(0, '275.310'), (1, '285.360')] [2023-10-12 23:02:29,334][44958] Updated weights for policy 0, policy_version 71650 (0.0008) [2023-10-12 23:02:29,703][44958] Updated weights for policy 0, policy_version 71660 (0.0010) [2023-10-12 23:02:29,805][44959] Updated weights for policy 1, policy_version 72010 (0.0007) [2023-10-12 23:02:30,070][44958] Updated weights for policy 0, policy_version 71670 (0.0008) [2023-10-12 23:02:30,170][44959] Updated weights for policy 1, policy_version 72020 (0.0007) [2023-10-12 23:02:30,448][44958] Updated weights for policy 0, policy_version 71680 (0.0007) [2023-10-12 23:02:30,539][44959] Updated weights for policy 1, policy_version 72030 (0.0008) [2023-10-12 23:02:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147161088. Throughput: 0: 1631.6, 1: 1640.8. Samples: 36796492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:31,443][43579] Avg episode reward: [(0, '276.170'), (1, '286.330')] [2023-10-12 23:02:34,749][44958] Updated weights for policy 0, policy_version 71690 (0.0008) [2023-10-12 23:02:34,757][44959] Updated weights for policy 1, policy_version 72040 (0.0007) [2023-10-12 23:02:35,116][44958] Updated weights for policy 0, policy_version 71700 (0.0007) [2023-10-12 23:02:35,135][44959] Updated weights for policy 1, policy_version 72050 (0.0007) [2023-10-12 23:02:35,492][44958] Updated weights for policy 0, policy_version 71710 (0.0007) [2023-10-12 23:02:35,496][44959] Updated weights for policy 1, policy_version 72060 (0.0007) [2023-10-12 23:02:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147226624. Throughput: 0: 1631.6, 1: 1643.4. Samples: 36807676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:36,443][43579] Avg episode reward: [(0, '278.160'), (1, '290.800')] [2023-10-12 23:02:39,682][44958] Updated weights for policy 0, policy_version 71720 (0.0007) [2023-10-12 23:02:39,695][44959] Updated weights for policy 1, policy_version 72070 (0.0008) [2023-10-12 23:02:40,054][44958] Updated weights for policy 0, policy_version 71730 (0.0007) [2023-10-12 23:02:40,065][44959] Updated weights for policy 1, policy_version 72080 (0.0008) [2023-10-12 23:02:40,420][44958] Updated weights for policy 0, policy_version 71740 (0.0010) [2023-10-12 23:02:40,424][44959] Updated weights for policy 1, policy_version 72090 (0.0009) [2023-10-12 23:02:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147292160. Throughput: 0: 1633.2, 1: 1641.0. Samples: 36826420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:41,443][43579] Avg episode reward: [(0, '268.070'), (1, '291.700')] [2023-10-12 23:02:44,496][44958] Updated weights for policy 0, policy_version 71750 (0.0008) [2023-10-12 23:02:44,565][44959] Updated weights for policy 1, policy_version 72100 (0.0008) [2023-10-12 23:02:44,861][44958] Updated weights for policy 0, policy_version 71760 (0.0008) [2023-10-12 23:02:44,933][44959] Updated weights for policy 1, policy_version 72110 (0.0008) [2023-10-12 23:02:45,231][44958] Updated weights for policy 0, policy_version 71770 (0.0010) [2023-10-12 23:02:45,308][44959] Updated weights for policy 1, policy_version 72120 (0.0008) [2023-10-12 23:02:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147357696. Throughput: 0: 1633.0, 1: 1644.2. Samples: 36845770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:46,443][43579] Avg episode reward: [(0, '268.590'), (1, '287.630')] [2023-10-12 23:02:49,548][44958] Updated weights for policy 0, policy_version 71780 (0.0008) [2023-10-12 23:02:49,553][44959] Updated weights for policy 1, policy_version 72130 (0.0008) [2023-10-12 23:02:49,919][44959] Updated weights for policy 1, policy_version 72140 (0.0007) [2023-10-12 23:02:49,938][44958] Updated weights for policy 0, policy_version 71790 (0.0008) [2023-10-12 23:02:50,292][44959] Updated weights for policy 1, policy_version 72150 (0.0007) [2023-10-12 23:02:50,318][44958] Updated weights for policy 0, policy_version 71800 (0.0007) [2023-10-12 23:02:50,654][44959] Updated weights for policy 1, policy_version 72160 (0.0007) [2023-10-12 23:02:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147423232. Throughput: 0: 1630.0, 1: 1640.4. Samples: 36856914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:51,443][43579] Avg episode reward: [(0, '266.450'), (1, '290.670')] [2023-10-12 23:02:54,485][44958] Updated weights for policy 0, policy_version 71810 (0.0009) [2023-10-12 23:02:54,852][44958] Updated weights for policy 0, policy_version 71820 (0.0010) [2023-10-12 23:02:54,975][44959] Updated weights for policy 1, policy_version 72170 (0.0008) [2023-10-12 23:02:55,228][44958] Updated weights for policy 0, policy_version 71830 (0.0009) [2023-10-12 23:02:55,344][44959] Updated weights for policy 1, policy_version 72180 (0.0008) [2023-10-12 23:02:55,595][44958] Updated weights for policy 0, policy_version 71840 (0.0009) [2023-10-12 23:02:55,714][44959] Updated weights for policy 1, policy_version 72190 (0.0008) [2023-10-12 23:02:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147488768. Throughput: 0: 1632.2, 1: 1640.0. Samples: 36875672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:02:56,443][43579] Avg episode reward: [(0, '269.080'), (1, '286.770')] [2023-10-12 23:02:59,710][44959] Updated weights for policy 1, policy_version 72200 (0.0009) [2023-10-12 23:02:59,817][44958] Updated weights for policy 0, policy_version 71850 (0.0009) [2023-10-12 23:03:00,074][44959] Updated weights for policy 1, policy_version 72210 (0.0009) [2023-10-12 23:03:00,186][44958] Updated weights for policy 0, policy_version 71860 (0.0009) [2023-10-12 23:03:00,439][44959] Updated weights for policy 1, policy_version 72220 (0.0007) [2023-10-12 23:03:00,560][44958] Updated weights for policy 0, policy_version 71870 (0.0008) [2023-10-12 23:03:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 147554304. Throughput: 0: 1624.5, 1: 1643.6. Samples: 36894696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:01,444][43579] Avg episode reward: [(0, '263.780'), (1, '282.700')] [2023-10-12 23:03:04,592][44959] Updated weights for policy 1, policy_version 72230 (0.0008) [2023-10-12 23:03:04,802][44958] Updated weights for policy 0, policy_version 71880 (0.0008) [2023-10-12 23:03:04,964][44959] Updated weights for policy 1, policy_version 72240 (0.0009) [2023-10-12 23:03:05,169][44958] Updated weights for policy 0, policy_version 71890 (0.0008) [2023-10-12 23:03:05,330][44959] Updated weights for policy 1, policy_version 72250 (0.0009) [2023-10-12 23:03:05,550][44958] Updated weights for policy 0, policy_version 71900 (0.0008) [2023-10-12 23:03:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147619840. Throughput: 0: 1623.4, 1: 1642.4. Samples: 36905936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:06,444][43579] Avg episode reward: [(0, '261.180'), (1, '281.550')] [2023-10-12 23:03:09,600][44959] Updated weights for policy 1, policy_version 72260 (0.0009) [2023-10-12 23:03:09,712][44958] Updated weights for policy 0, policy_version 71910 (0.0008) [2023-10-12 23:03:09,976][44959] Updated weights for policy 1, policy_version 72270 (0.0008) [2023-10-12 23:03:10,078][44958] Updated weights for policy 0, policy_version 71920 (0.0008) [2023-10-12 23:03:10,345][44959] Updated weights for policy 1, policy_version 72280 (0.0007) [2023-10-12 23:03:10,456][44958] Updated weights for policy 0, policy_version 71930 (0.0009) [2023-10-12 23:03:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147685376. Throughput: 0: 1636.0, 1: 1643.1. Samples: 36924938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:11,444][43579] Avg episode reward: [(0, '273.380'), (1, '279.450')] [2023-10-12 23:03:14,463][44958] Updated weights for policy 0, policy_version 71940 (0.0009) [2023-10-12 23:03:14,530][44959] Updated weights for policy 1, policy_version 72290 (0.0008) [2023-10-12 23:03:14,838][44958] Updated weights for policy 0, policy_version 71950 (0.0009) [2023-10-12 23:03:14,892][44959] Updated weights for policy 1, policy_version 72300 (0.0008) [2023-10-12 23:03:15,209][44958] Updated weights for policy 0, policy_version 71960 (0.0009) [2023-10-12 23:03:15,261][44959] Updated weights for policy 1, policy_version 72310 (0.0007) [2023-10-12 23:03:15,628][44959] Updated weights for policy 1, policy_version 72320 (0.0009) [2023-10-12 23:03:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147750912. Throughput: 0: 1639.8, 1: 1641.9. Samples: 36944168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:16,444][43579] Avg episode reward: [(0, '271.210'), (1, '283.910')] [2023-10-12 23:03:19,375][44958] Updated weights for policy 0, policy_version 71970 (0.0010) [2023-10-12 23:03:19,757][44958] Updated weights for policy 0, policy_version 71980 (0.0008) [2023-10-12 23:03:19,964][44959] Updated weights for policy 1, policy_version 72330 (0.0007) [2023-10-12 23:03:20,119][44958] Updated weights for policy 0, policy_version 71990 (0.0009) [2023-10-12 23:03:20,333][44959] Updated weights for policy 1, policy_version 72340 (0.0007) [2023-10-12 23:03:20,494][44958] Updated weights for policy 0, policy_version 72000 (0.0007) [2023-10-12 23:03:20,702][44959] Updated weights for policy 1, policy_version 72350 (0.0011) [2023-10-12 23:03:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147816448. Throughput: 0: 1645.1, 1: 1638.7. Samples: 36955446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:21,443][43579] Avg episode reward: [(0, '265.160'), (1, '282.890')] [2023-10-12 23:03:24,678][44958] Updated weights for policy 0, policy_version 72010 (0.0007) [2023-10-12 23:03:24,787][44959] Updated weights for policy 1, policy_version 72360 (0.0008) [2023-10-12 23:03:25,050][44958] Updated weights for policy 0, policy_version 72020 (0.0007) [2023-10-12 23:03:25,148][44959] Updated weights for policy 1, policy_version 72370 (0.0007) [2023-10-12 23:03:25,427][44958] Updated weights for policy 0, policy_version 72030 (0.0008) [2023-10-12 23:03:25,504][44959] Updated weights for policy 1, policy_version 72380 (0.0008) [2023-10-12 23:03:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147881984. Throughput: 0: 1644.8, 1: 1638.9. Samples: 36974184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:26,443][43579] Avg episode reward: [(0, '258.190'), (1, '285.800')] [2023-10-12 23:03:29,541][44958] Updated weights for policy 0, policy_version 72040 (0.0010) [2023-10-12 23:03:29,697][44959] Updated weights for policy 1, policy_version 72390 (0.0008) [2023-10-12 23:03:29,904][44958] Updated weights for policy 0, policy_version 72050 (0.0009) [2023-10-12 23:03:30,063][44959] Updated weights for policy 1, policy_version 72400 (0.0007) [2023-10-12 23:03:30,276][44958] Updated weights for policy 0, policy_version 72060 (0.0009) [2023-10-12 23:03:30,438][44959] Updated weights for policy 1, policy_version 72410 (0.0008) [2023-10-12 23:03:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 147947520. Throughput: 0: 1636.5, 1: 1637.6. Samples: 36993106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:31,444][43579] Avg episode reward: [(0, '258.440'), (1, '283.070')] [2023-10-12 23:03:34,659][44959] Updated weights for policy 1, policy_version 72420 (0.0009) [2023-10-12 23:03:34,679][44958] Updated weights for policy 0, policy_version 72070 (0.0007) [2023-10-12 23:03:35,034][44959] Updated weights for policy 1, policy_version 72430 (0.0007) [2023-10-12 23:03:35,058][44958] Updated weights for policy 0, policy_version 72080 (0.0007) [2023-10-12 23:03:35,394][44959] Updated weights for policy 1, policy_version 72440 (0.0009) [2023-10-12 23:03:35,429][44958] Updated weights for policy 0, policy_version 72090 (0.0008) [2023-10-12 23:03:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148013056. Throughput: 0: 1635.2, 1: 1638.7. Samples: 37004236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:36,444][43579] Avg episode reward: [(0, '250.010'), (1, '281.770')] [2023-10-12 23:03:39,470][44958] Updated weights for policy 0, policy_version 72100 (0.0008) [2023-10-12 23:03:39,581][44959] Updated weights for policy 1, policy_version 72450 (0.0008) [2023-10-12 23:03:39,853][44958] Updated weights for policy 0, policy_version 72110 (0.0007) [2023-10-12 23:03:39,956][44959] Updated weights for policy 1, policy_version 72460 (0.0007) [2023-10-12 23:03:40,222][44958] Updated weights for policy 0, policy_version 72120 (0.0010) [2023-10-12 23:03:40,325][44959] Updated weights for policy 1, policy_version 72470 (0.0008) [2023-10-12 23:03:40,693][44959] Updated weights for policy 1, policy_version 72480 (0.0009) [2023-10-12 23:03:41,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148078592. Throughput: 0: 1640.3, 1: 1634.6. Samples: 37023044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:41,443][43579] Avg episode reward: [(0, '247.950'), (1, '284.450')] [2023-10-12 23:03:44,613][44958] Updated weights for policy 0, policy_version 72130 (0.0008) [2023-10-12 23:03:44,836][44959] Updated weights for policy 1, policy_version 72490 (0.0008) [2023-10-12 23:03:44,980][44958] Updated weights for policy 0, policy_version 72140 (0.0008) [2023-10-12 23:03:45,198][44959] Updated weights for policy 1, policy_version 72500 (0.0008) [2023-10-12 23:03:45,355][44958] Updated weights for policy 0, policy_version 72150 (0.0008) [2023-10-12 23:03:45,561][44959] Updated weights for policy 1, policy_version 72510 (0.0007) [2023-10-12 23:03:45,728][44958] Updated weights for policy 0, policy_version 72160 (0.0009) [2023-10-12 23:03:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148144128. Throughput: 0: 1641.7, 1: 1636.0. Samples: 37042192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:46,443][43579] Avg episode reward: [(0, '247.240'), (1, '279.580')] [2023-10-12 23:03:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000072160_73891840.pth... [2023-10-12 23:03:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000072512_74252288.pth... [2023-10-12 23:03:46,485][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000070624_72318976.pth [2023-10-12 23:03:46,493][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000070976_72679424.pth [2023-10-12 23:03:49,721][44959] Updated weights for policy 1, policy_version 72520 (0.0008) [2023-10-12 23:03:49,853][44958] Updated weights for policy 0, policy_version 72170 (0.0008) [2023-10-12 23:03:50,082][44959] Updated weights for policy 1, policy_version 72530 (0.0007) [2023-10-12 23:03:50,226][44958] Updated weights for policy 0, policy_version 72180 (0.0008) [2023-10-12 23:03:50,449][44959] Updated weights for policy 1, policy_version 72540 (0.0007) [2023-10-12 23:03:50,596][44958] Updated weights for policy 0, policy_version 72190 (0.0009) [2023-10-12 23:03:51,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 148209664. Throughput: 0: 1642.8, 1: 1632.4. Samples: 37053320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:51,444][43579] Avg episode reward: [(0, '257.880'), (1, '282.020')] [2023-10-12 23:03:54,679][44958] Updated weights for policy 0, policy_version 72200 (0.0008) [2023-10-12 23:03:54,911][44959] Updated weights for policy 1, policy_version 72550 (0.0008) [2023-10-12 23:03:55,056][44958] Updated weights for policy 0, policy_version 72210 (0.0009) [2023-10-12 23:03:55,273][44959] Updated weights for policy 1, policy_version 72560 (0.0007) [2023-10-12 23:03:55,434][44958] Updated weights for policy 0, policy_version 72220 (0.0009) [2023-10-12 23:03:55,641][44959] Updated weights for policy 1, policy_version 72570 (0.0007) [2023-10-12 23:03:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148275200. Throughput: 0: 1640.8, 1: 1633.0. Samples: 37072262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:03:56,443][43579] Avg episode reward: [(0, '253.450'), (1, '282.490')] [2023-10-12 23:03:59,638][44959] Updated weights for policy 1, policy_version 72580 (0.0008) [2023-10-12 23:03:59,666][44958] Updated weights for policy 0, policy_version 72230 (0.0008) [2023-10-12 23:04:00,002][44959] Updated weights for policy 1, policy_version 72590 (0.0007) [2023-10-12 23:04:00,035][44958] Updated weights for policy 0, policy_version 72240 (0.0007) [2023-10-12 23:04:00,378][44959] Updated weights for policy 1, policy_version 72600 (0.0008) [2023-10-12 23:04:00,405][44958] Updated weights for policy 0, policy_version 72250 (0.0007) [2023-10-12 23:04:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148340736. Throughput: 0: 1636.5, 1: 1632.9. Samples: 37091294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:04:01,444][43579] Avg episode reward: [(0, '258.330'), (1, '278.570')] [2023-10-12 23:04:04,513][44959] Updated weights for policy 1, policy_version 72610 (0.0009) [2023-10-12 23:04:04,641][44958] Updated weights for policy 0, policy_version 72260 (0.0008) [2023-10-12 23:04:04,919][44959] Updated weights for policy 1, policy_version 72620 (0.0008) [2023-10-12 23:04:05,011][44958] Updated weights for policy 0, policy_version 72270 (0.0009) [2023-10-12 23:04:05,290][44959] Updated weights for policy 1, policy_version 72630 (0.0009) [2023-10-12 23:04:05,384][44958] Updated weights for policy 0, policy_version 72280 (0.0010) [2023-10-12 23:04:05,654][44959] Updated weights for policy 1, policy_version 72640 (0.0008) [2023-10-12 23:04:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148406272. Throughput: 0: 1635.0, 1: 1637.4. Samples: 37102704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:04:06,443][43579] Avg episode reward: [(0, '261.710'), (1, '280.680')] [2023-10-12 23:04:09,563][44959] Updated weights for policy 1, policy_version 72650 (0.0009) [2023-10-12 23:04:09,601][44958] Updated weights for policy 0, policy_version 72290 (0.0008) [2023-10-12 23:04:09,933][44959] Updated weights for policy 1, policy_version 72660 (0.0010) [2023-10-12 23:04:09,965][44958] Updated weights for policy 0, policy_version 72300 (0.0007) [2023-10-12 23:04:10,304][44959] Updated weights for policy 1, policy_version 72670 (0.0008) [2023-10-12 23:04:10,343][44958] Updated weights for policy 0, policy_version 72310 (0.0009) [2023-10-12 23:04:10,720][44958] Updated weights for policy 0, policy_version 72320 (0.0011) [2023-10-12 23:04:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148471808. Throughput: 0: 1637.6, 1: 1631.4. Samples: 37121292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:04:11,443][43579] Avg episode reward: [(0, '263.880'), (1, '277.940')] [2023-10-12 23:04:14,802][44959] Updated weights for policy 1, policy_version 72680 (0.0008) [2023-10-12 23:04:14,944][44958] Updated weights for policy 0, policy_version 72330 (0.0009) [2023-10-12 23:04:15,177][44959] Updated weights for policy 1, policy_version 72690 (0.0009) [2023-10-12 23:04:15,330][44958] Updated weights for policy 0, policy_version 72340 (0.0009) [2023-10-12 23:04:15,546][44959] Updated weights for policy 1, policy_version 72700 (0.0009) [2023-10-12 23:04:15,705][44958] Updated weights for policy 0, policy_version 72350 (0.0008) [2023-10-12 23:04:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148537344. Throughput: 0: 1630.8, 1: 1639.7. Samples: 37140278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:04:16,443][43579] Avg episode reward: [(0, '261.910'), (1, '278.010')] [2023-10-12 23:04:19,727][44959] Updated weights for policy 1, policy_version 72710 (0.0009) [2023-10-12 23:04:19,895][44958] Updated weights for policy 0, policy_version 72360 (0.0008) [2023-10-12 23:04:20,102][44959] Updated weights for policy 1, policy_version 72720 (0.0008) [2023-10-12 23:04:20,271][44958] Updated weights for policy 0, policy_version 72370 (0.0007) [2023-10-12 23:04:20,472][44959] Updated weights for policy 1, policy_version 72730 (0.0008) [2023-10-12 23:04:20,647][44958] Updated weights for policy 0, policy_version 72380 (0.0008) [2023-10-12 23:04:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148602880. Throughput: 0: 1633.6, 1: 1640.8. Samples: 37151584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:04:21,443][43579] Avg episode reward: [(0, '260.660'), (1, '278.060')] [2023-10-12 23:04:24,513][44959] Updated weights for policy 1, policy_version 72740 (0.0008) [2023-10-12 23:04:24,884][44959] Updated weights for policy 1, policy_version 72750 (0.0007) [2023-10-12 23:04:24,905][44958] Updated weights for policy 0, policy_version 72390 (0.0009) [2023-10-12 23:04:25,246][44959] Updated weights for policy 1, policy_version 72760 (0.0008) [2023-10-12 23:04:25,293][44958] Updated weights for policy 0, policy_version 72400 (0.0009) [2023-10-12 23:04:25,663][44958] Updated weights for policy 0, policy_version 72410 (0.0008) [2023-10-12 23:04:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148668416. Throughput: 0: 1639.4, 1: 1638.7. Samples: 37170558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:04:26,443][43579] Avg episode reward: [(0, '267.620'), (1, '284.210')] [2023-10-12 23:04:29,546][44959] Updated weights for policy 1, policy_version 72770 (0.0008) [2023-10-12 23:04:29,762][44958] Updated weights for policy 0, policy_version 72420 (0.0009) [2023-10-12 23:04:29,918][44959] Updated weights for policy 1, policy_version 72780 (0.0008) [2023-10-12 23:04:30,129][44958] Updated weights for policy 0, policy_version 72430 (0.0008) [2023-10-12 23:04:30,296][44959] Updated weights for policy 1, policy_version 72790 (0.0009) [2023-10-12 23:04:30,509][44958] Updated weights for policy 0, policy_version 72440 (0.0008) [2023-10-12 23:04:30,665][44959] Updated weights for policy 1, policy_version 72800 (0.0009) [2023-10-12 23:04:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148733952. Throughput: 0: 1630.8, 1: 1636.0. Samples: 37189200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:04:31,443][43579] Avg episode reward: [(0, '261.640'), (1, '282.630')] [2023-10-12 23:04:34,767][44958] Updated weights for policy 0, policy_version 72450 (0.0009) [2023-10-12 23:04:34,912][44959] Updated weights for policy 1, policy_version 72810 (0.0007) [2023-10-12 23:04:35,139][44958] Updated weights for policy 0, policy_version 72460 (0.0008) [2023-10-12 23:04:35,280][44959] Updated weights for policy 1, policy_version 72820 (0.0009) [2023-10-12 23:04:35,509][44958] Updated weights for policy 0, policy_version 72470 (0.0010) [2023-10-12 23:04:35,650][44959] Updated weights for policy 1, policy_version 72830 (0.0010) [2023-10-12 23:04:35,886][44958] Updated weights for policy 0, policy_version 72480 (0.0010) [2023-10-12 23:04:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148799488. Throughput: 0: 1629.0, 1: 1641.2. Samples: 37200480. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:04:36,443][43579] Avg episode reward: [(0, '260.400'), (1, '282.060')] [2023-10-12 23:04:39,969][44959] Updated weights for policy 1, policy_version 72840 (0.0007) [2023-10-12 23:04:40,114][44958] Updated weights for policy 0, policy_version 72490 (0.0009) [2023-10-12 23:04:40,332][44959] Updated weights for policy 1, policy_version 72850 (0.0008) [2023-10-12 23:04:40,486][44958] Updated weights for policy 0, policy_version 72500 (0.0007) [2023-10-12 23:04:40,702][44959] Updated weights for policy 1, policy_version 72860 (0.0007) [2023-10-12 23:04:40,850][44958] Updated weights for policy 0, policy_version 72510 (0.0007) [2023-10-12 23:04:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148865024. Throughput: 0: 1633.1, 1: 1640.2. Samples: 37219558. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:04:41,443][43579] Avg episode reward: [(0, '263.020'), (1, '285.900')] [2023-10-12 23:04:44,795][44959] Updated weights for policy 1, policy_version 72870 (0.0007) [2023-10-12 23:04:45,161][44959] Updated weights for policy 1, policy_version 72880 (0.0008) [2023-10-12 23:04:45,183][44958] Updated weights for policy 0, policy_version 72520 (0.0008) [2023-10-12 23:04:45,533][44959] Updated weights for policy 1, policy_version 72890 (0.0009) [2023-10-12 23:04:45,546][44958] Updated weights for policy 0, policy_version 72530 (0.0009) [2023-10-12 23:04:45,921][44958] Updated weights for policy 0, policy_version 72540 (0.0009) [2023-10-12 23:04:46,442][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148930560. Throughput: 0: 1619.6, 1: 1642.8. Samples: 37238106. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:04:46,443][43579] Avg episode reward: [(0, '265.210'), (1, '285.200')] [2023-10-12 23:04:49,775][44959] Updated weights for policy 1, policy_version 72900 (0.0008) [2023-10-12 23:04:50,031][44958] Updated weights for policy 0, policy_version 72550 (0.0007) [2023-10-12 23:04:50,182][44959] Updated weights for policy 1, policy_version 72910 (0.0008) [2023-10-12 23:04:50,413][44958] Updated weights for policy 0, policy_version 72560 (0.0009) [2023-10-12 23:04:50,552][44959] Updated weights for policy 1, policy_version 72920 (0.0007) [2023-10-12 23:04:50,789][44958] Updated weights for policy 0, policy_version 72570 (0.0009) [2023-10-12 23:04:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 148996096. Throughput: 0: 1618.4, 1: 1637.4. Samples: 37249216. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:04:51,443][43579] Avg episode reward: [(0, '258.780'), (1, '284.880')] [2023-10-12 23:04:54,597][44959] Updated weights for policy 1, policy_version 72930 (0.0007) [2023-10-12 23:04:54,890][44958] Updated weights for policy 0, policy_version 72580 (0.0008) [2023-10-12 23:04:54,956][44959] Updated weights for policy 1, policy_version 72940 (0.0007) [2023-10-12 23:04:55,266][44958] Updated weights for policy 0, policy_version 72590 (0.0008) [2023-10-12 23:04:55,330][44959] Updated weights for policy 1, policy_version 72950 (0.0008) [2023-10-12 23:04:55,633][44958] Updated weights for policy 0, policy_version 72600 (0.0008) [2023-10-12 23:04:55,689][44959] Updated weights for policy 1, policy_version 72960 (0.0009) [2023-10-12 23:04:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149061632. Throughput: 0: 1627.2, 1: 1646.5. Samples: 37268610. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:04:56,443][43579] Avg episode reward: [(0, '267.500'), (1, '281.870')] [2023-10-12 23:04:59,845][44958] Updated weights for policy 0, policy_version 72610 (0.0008) [2023-10-12 23:05:00,018][44959] Updated weights for policy 1, policy_version 72970 (0.0008) [2023-10-12 23:05:00,218][44958] Updated weights for policy 0, policy_version 72620 (0.0007) [2023-10-12 23:05:00,385][44959] Updated weights for policy 1, policy_version 72980 (0.0007) [2023-10-12 23:05:00,590][44958] Updated weights for policy 0, policy_version 72630 (0.0008) [2023-10-12 23:05:00,756][44959] Updated weights for policy 1, policy_version 72990 (0.0009) [2023-10-12 23:05:00,955][44958] Updated weights for policy 0, policy_version 72640 (0.0008) [2023-10-12 23:05:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149127168. Throughput: 0: 1629.6, 1: 1635.2. Samples: 37287192. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:05:01,443][43579] Avg episode reward: [(0, '265.210'), (1, '287.020')] [2023-10-12 23:05:04,912][44959] Updated weights for policy 1, policy_version 73000 (0.0010) [2023-10-12 23:05:05,062][44958] Updated weights for policy 0, policy_version 72650 (0.0007) [2023-10-12 23:05:05,280][44959] Updated weights for policy 1, policy_version 73010 (0.0009) [2023-10-12 23:05:05,440][44958] Updated weights for policy 0, policy_version 72660 (0.0009) [2023-10-12 23:05:05,655][44959] Updated weights for policy 1, policy_version 73020 (0.0009) [2023-10-12 23:05:05,810][44958] Updated weights for policy 0, policy_version 72670 (0.0009) [2023-10-12 23:05:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149192704. Throughput: 0: 1629.8, 1: 1637.4. Samples: 37298608. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:05:06,443][43579] Avg episode reward: [(0, '267.240'), (1, '285.930')] [2023-10-12 23:05:09,678][44959] Updated weights for policy 1, policy_version 73030 (0.0007) [2023-10-12 23:05:10,042][44959] Updated weights for policy 1, policy_version 73040 (0.0008) [2023-10-12 23:05:10,074][44958] Updated weights for policy 0, policy_version 72680 (0.0009) [2023-10-12 23:05:10,410][44959] Updated weights for policy 1, policy_version 73050 (0.0007) [2023-10-12 23:05:10,442][44958] Updated weights for policy 0, policy_version 72690 (0.0008) [2023-10-12 23:05:10,804][44958] Updated weights for policy 0, policy_version 72700 (0.0009) [2023-10-12 23:05:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149258240. Throughput: 0: 1628.5, 1: 1639.3. Samples: 37317610. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:05:11,444][43579] Avg episode reward: [(0, '265.640'), (1, '284.530')] [2023-10-12 23:05:14,748][44959] Updated weights for policy 1, policy_version 73060 (0.0008) [2023-10-12 23:05:14,957][44958] Updated weights for policy 0, policy_version 72710 (0.0007) [2023-10-12 23:05:15,110][44959] Updated weights for policy 1, policy_version 73070 (0.0007) [2023-10-12 23:05:15,325][44958] Updated weights for policy 0, policy_version 72720 (0.0007) [2023-10-12 23:05:15,472][44959] Updated weights for policy 1, policy_version 73080 (0.0008) [2023-10-12 23:05:15,687][44958] Updated weights for policy 0, policy_version 72730 (0.0008) [2023-10-12 23:05:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149323776. Throughput: 0: 1631.1, 1: 1637.0. Samples: 37336266. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:05:16,443][43579] Avg episode reward: [(0, '268.710'), (1, '287.020')] [2023-10-12 23:05:19,488][44959] Updated weights for policy 1, policy_version 73090 (0.0009) [2023-10-12 23:05:19,851][44959] Updated weights for policy 1, policy_version 73100 (0.0008) [2023-10-12 23:05:19,891][44958] Updated weights for policy 0, policy_version 72740 (0.0008) [2023-10-12 23:05:20,221][44959] Updated weights for policy 1, policy_version 73110 (0.0008) [2023-10-12 23:05:20,260][44958] Updated weights for policy 0, policy_version 72750 (0.0009) [2023-10-12 23:05:20,594][44959] Updated weights for policy 1, policy_version 73120 (0.0008) [2023-10-12 23:05:20,636][44958] Updated weights for policy 0, policy_version 72760 (0.0009) [2023-10-12 23:05:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 149389312. Throughput: 0: 1632.4, 1: 1637.3. Samples: 37347616. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:05:21,444][43579] Avg episode reward: [(0, '275.940'), (1, '284.620')] [2023-10-12 23:05:24,956][44959] Updated weights for policy 1, policy_version 73130 (0.0009) [2023-10-12 23:05:24,994][44958] Updated weights for policy 0, policy_version 72770 (0.0009) [2023-10-12 23:05:25,332][44959] Updated weights for policy 1, policy_version 73140 (0.0009) [2023-10-12 23:05:25,362][44958] Updated weights for policy 0, policy_version 72780 (0.0008) [2023-10-12 23:05:25,691][44959] Updated weights for policy 1, policy_version 73150 (0.0007) [2023-10-12 23:05:25,728][44958] Updated weights for policy 0, policy_version 72790 (0.0009) [2023-10-12 23:05:26,100][44958] Updated weights for policy 0, policy_version 72800 (0.0010) [2023-10-12 23:05:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149454848. Throughput: 0: 1636.2, 1: 1638.8. Samples: 37366932. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:05:26,444][43579] Avg episode reward: [(0, '275.460'), (1, '282.860')] [2023-10-12 23:05:29,929][44959] Updated weights for policy 1, policy_version 73160 (0.0008) [2023-10-12 23:05:30,220][44958] Updated weights for policy 0, policy_version 72810 (0.0007) [2023-10-12 23:05:30,290][44959] Updated weights for policy 1, policy_version 73170 (0.0007) [2023-10-12 23:05:30,589][44958] Updated weights for policy 0, policy_version 72820 (0.0007) [2023-10-12 23:05:30,660][44959] Updated weights for policy 1, policy_version 73180 (0.0008) [2023-10-12 23:05:30,970][44958] Updated weights for policy 0, policy_version 72830 (0.0010) [2023-10-12 23:05:31,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149520384. Throughput: 0: 1634.4, 1: 1633.3. Samples: 37385156. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:05:31,443][43579] Avg episode reward: [(0, '277.400'), (1, '284.000')] [2023-10-12 23:05:34,980][44959] Updated weights for policy 1, policy_version 73190 (0.0009) [2023-10-12 23:05:35,037][44958] Updated weights for policy 0, policy_version 72840 (0.0008) [2023-10-12 23:05:35,360][44959] Updated weights for policy 1, policy_version 73200 (0.0008) [2023-10-12 23:05:35,407][44958] Updated weights for policy 0, policy_version 72850 (0.0008) [2023-10-12 23:05:35,728][44959] Updated weights for policy 1, policy_version 73210 (0.0008) [2023-10-12 23:05:35,782][44958] Updated weights for policy 0, policy_version 72860 (0.0007) [2023-10-12 23:05:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149585920. Throughput: 0: 1638.4, 1: 1635.0. Samples: 37396520. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:05:36,443][43579] Avg episode reward: [(0, '270.160'), (1, '281.780')] [2023-10-12 23:05:39,920][44959] Updated weights for policy 1, policy_version 73220 (0.0009) [2023-10-12 23:05:40,033][44958] Updated weights for policy 0, policy_version 72870 (0.0008) [2023-10-12 23:05:40,281][44959] Updated weights for policy 1, policy_version 73230 (0.0010) [2023-10-12 23:05:40,399][44958] Updated weights for policy 0, policy_version 72880 (0.0009) [2023-10-12 23:05:40,652][44959] Updated weights for policy 1, policy_version 73240 (0.0007) [2023-10-12 23:05:40,767][44958] Updated weights for policy 0, policy_version 72890 (0.0008) [2023-10-12 23:05:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149651456. Throughput: 0: 1638.3, 1: 1633.4. Samples: 37415834. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:05:41,444][43579] Avg episode reward: [(0, '271.480'), (1, '282.130')] [2023-10-12 23:05:44,697][44959] Updated weights for policy 1, policy_version 73250 (0.0007) [2023-10-12 23:05:44,909][44958] Updated weights for policy 0, policy_version 72900 (0.0009) [2023-10-12 23:05:45,073][44959] Updated weights for policy 1, policy_version 73260 (0.0009) [2023-10-12 23:05:45,273][44958] Updated weights for policy 0, policy_version 72910 (0.0009) [2023-10-12 23:05:45,428][44959] Updated weights for policy 1, policy_version 73270 (0.0009) [2023-10-12 23:05:45,643][44958] Updated weights for policy 0, policy_version 72920 (0.0009) [2023-10-12 23:05:45,793][44959] Updated weights for policy 1, policy_version 73280 (0.0009) [2023-10-12 23:05:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149716992. Throughput: 0: 1633.0, 1: 1632.4. Samples: 37434132. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:05:46,443][43579] Avg episode reward: [(0, '270.810'), (1, '280.880')] [2023-10-12 23:05:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000073280_75038720.pth... [2023-10-12 23:05:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000072928_74678272.pth... [2023-10-12 23:05:46,485][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000071744_73465856.pth [2023-10-12 23:05:46,491][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000071392_73105408.pth [2023-10-12 23:05:49,856][44958] Updated weights for policy 0, policy_version 72930 (0.0007) [2023-10-12 23:05:49,942][44959] Updated weights for policy 1, policy_version 73290 (0.0009) [2023-10-12 23:05:50,230][44958] Updated weights for policy 0, policy_version 72940 (0.0008) [2023-10-12 23:05:50,309][44959] Updated weights for policy 1, policy_version 73300 (0.0008) [2023-10-12 23:05:50,599][44958] Updated weights for policy 0, policy_version 72950 (0.0008) [2023-10-12 23:05:50,665][44959] Updated weights for policy 1, policy_version 73310 (0.0008) [2023-10-12 23:05:50,973][44958] Updated weights for policy 0, policy_version 72960 (0.0008) [2023-10-12 23:05:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149782528. Throughput: 0: 1629.7, 1: 1629.6. Samples: 37445276. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:05:51,444][43579] Avg episode reward: [(0, '269.440'), (1, '280.640')] [2023-10-12 23:05:54,800][44959] Updated weights for policy 1, policy_version 73320 (0.0008) [2023-10-12 23:05:55,100][44958] Updated weights for policy 0, policy_version 72970 (0.0007) [2023-10-12 23:05:55,165][44959] Updated weights for policy 1, policy_version 73330 (0.0008) [2023-10-12 23:05:55,484][44958] Updated weights for policy 0, policy_version 72980 (0.0008) [2023-10-12 23:05:55,540][44959] Updated weights for policy 1, policy_version 73340 (0.0009) [2023-10-12 23:05:55,844][44958] Updated weights for policy 0, policy_version 72990 (0.0008) [2023-10-12 23:05:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149848064. Throughput: 0: 1634.7, 1: 1633.1. Samples: 37464658. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:05:56,444][43579] Avg episode reward: [(0, '272.320'), (1, '278.820')] [2023-10-12 23:05:59,801][44959] Updated weights for policy 1, policy_version 73350 (0.0010) [2023-10-12 23:05:59,993][44958] Updated weights for policy 0, policy_version 73000 (0.0007) [2023-10-12 23:06:00,170][44959] Updated weights for policy 1, policy_version 73360 (0.0008) [2023-10-12 23:06:00,369][44958] Updated weights for policy 0, policy_version 73010 (0.0008) [2023-10-12 23:06:00,535][44959] Updated weights for policy 1, policy_version 73370 (0.0009) [2023-10-12 23:06:00,745][44958] Updated weights for policy 0, policy_version 73020 (0.0009) [2023-10-12 23:06:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149913600. Throughput: 0: 1632.4, 1: 1636.6. Samples: 37483368. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:06:01,443][43579] Avg episode reward: [(0, '274.970'), (1, '277.490')] [2023-10-12 23:06:04,647][44959] Updated weights for policy 1, policy_version 73380 (0.0008) [2023-10-12 23:06:05,010][44959] Updated weights for policy 1, policy_version 73390 (0.0008) [2023-10-12 23:06:05,040][44958] Updated weights for policy 0, policy_version 73030 (0.0008) [2023-10-12 23:06:05,381][44959] Updated weights for policy 1, policy_version 73400 (0.0007) [2023-10-12 23:06:05,411][44958] Updated weights for policy 0, policy_version 73040 (0.0007) [2023-10-12 23:06:05,778][44958] Updated weights for policy 0, policy_version 73050 (0.0008) [2023-10-12 23:06:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 149979136. Throughput: 0: 1635.6, 1: 1635.2. Samples: 37494802. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:06:06,444][43579] Avg episode reward: [(0, '281.350'), (1, '282.090')] [2023-10-12 23:06:09,601][44959] Updated weights for policy 1, policy_version 73410 (0.0008) [2023-10-12 23:06:09,968][44959] Updated weights for policy 1, policy_version 73420 (0.0009) [2023-10-12 23:06:10,029][44958] Updated weights for policy 0, policy_version 73060 (0.0008) [2023-10-12 23:06:10,329][44959] Updated weights for policy 1, policy_version 73430 (0.0009) [2023-10-12 23:06:10,398][44958] Updated weights for policy 0, policy_version 73070 (0.0007) [2023-10-12 23:06:10,698][44959] Updated weights for policy 1, policy_version 73440 (0.0008) [2023-10-12 23:06:10,772][44958] Updated weights for policy 0, policy_version 73080 (0.0009) [2023-10-12 23:06:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150044672. Throughput: 0: 1638.9, 1: 1641.2. Samples: 37514534. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-12 23:06:11,444][43579] Avg episode reward: [(0, '279.500'), (1, '285.010')] [2023-10-12 23:06:14,859][44959] Updated weights for policy 1, policy_version 73450 (0.0008) [2023-10-12 23:06:14,893][44958] Updated weights for policy 0, policy_version 73090 (0.0008) [2023-10-12 23:06:15,215][44959] Updated weights for policy 1, policy_version 73460 (0.0009) [2023-10-12 23:06:15,252][44958] Updated weights for policy 0, policy_version 73100 (0.0007) [2023-10-12 23:06:15,593][44959] Updated weights for policy 1, policy_version 73470 (0.0008) [2023-10-12 23:06:15,627][44958] Updated weights for policy 0, policy_version 73110 (0.0008) [2023-10-12 23:06:15,997][44958] Updated weights for policy 0, policy_version 73120 (0.0008) [2023-10-12 23:06:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150110208. Throughput: 0: 1637.0, 1: 1645.8. Samples: 37532880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:06:16,443][43579] Avg episode reward: [(0, '274.870'), (1, '284.580')] [2023-10-12 23:06:19,885][44959] Updated weights for policy 1, policy_version 73480 (0.0007) [2023-10-12 23:06:20,170][44958] Updated weights for policy 0, policy_version 73130 (0.0007) [2023-10-12 23:06:20,261][44959] Updated weights for policy 1, policy_version 73490 (0.0009) [2023-10-12 23:06:20,551][44958] Updated weights for policy 0, policy_version 73140 (0.0008) [2023-10-12 23:06:20,635][44959] Updated weights for policy 1, policy_version 73500 (0.0008) [2023-10-12 23:06:20,924][44958] Updated weights for policy 0, policy_version 73150 (0.0007) [2023-10-12 23:06:21,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150175744. Throughput: 0: 1637.4, 1: 1646.4. Samples: 37544294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:06:21,444][43579] Avg episode reward: [(0, '271.690'), (1, '287.810')] [2023-10-12 23:06:24,708][44959] Updated weights for policy 1, policy_version 73510 (0.0008) [2023-10-12 23:06:25,040][44958] Updated weights for policy 0, policy_version 73160 (0.0007) [2023-10-12 23:06:25,075][44959] Updated weights for policy 1, policy_version 73520 (0.0009) [2023-10-12 23:06:25,414][44958] Updated weights for policy 0, policy_version 73170 (0.0007) [2023-10-12 23:06:25,434][44959] Updated weights for policy 1, policy_version 73530 (0.0008) [2023-10-12 23:06:25,770][44958] Updated weights for policy 0, policy_version 73180 (0.0008) [2023-10-12 23:06:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150241280. Throughput: 0: 1633.2, 1: 1646.5. Samples: 37563420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:06:26,443][43579] Avg episode reward: [(0, '271.240'), (1, '290.420')] [2023-10-12 23:06:29,559][44959] Updated weights for policy 1, policy_version 73540 (0.0007) [2023-10-12 23:06:29,933][44959] Updated weights for policy 1, policy_version 73550 (0.0008) [2023-10-12 23:06:30,179][44958] Updated weights for policy 0, policy_version 73190 (0.0008) [2023-10-12 23:06:30,305][44959] Updated weights for policy 1, policy_version 73560 (0.0008) [2023-10-12 23:06:30,557][44958] Updated weights for policy 0, policy_version 73200 (0.0008) [2023-10-12 23:06:30,929][44958] Updated weights for policy 0, policy_version 73210 (0.0007) [2023-10-12 23:06:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150306816. Throughput: 0: 1632.6, 1: 1649.5. Samples: 37581824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:06:31,443][43579] Avg episode reward: [(0, '274.260'), (1, '291.640')] [2023-10-12 23:06:34,356][44959] Updated weights for policy 1, policy_version 73570 (0.0008) [2023-10-12 23:06:34,720][44959] Updated weights for policy 1, policy_version 73580 (0.0007) [2023-10-12 23:06:34,954][44958] Updated weights for policy 0, policy_version 73220 (0.0008) [2023-10-12 23:06:35,089][44959] Updated weights for policy 1, policy_version 73590 (0.0008) [2023-10-12 23:06:35,314][44958] Updated weights for policy 0, policy_version 73230 (0.0008) [2023-10-12 23:06:35,464][44959] Updated weights for policy 1, policy_version 73600 (0.0007) [2023-10-12 23:06:35,679][44958] Updated weights for policy 0, policy_version 73240 (0.0009) [2023-10-12 23:06:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150372352. Throughput: 0: 1639.5, 1: 1651.5. Samples: 37593368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:06:36,443][43579] Avg episode reward: [(0, '273.240'), (1, '290.920')] [2023-10-12 23:06:39,745][44959] Updated weights for policy 1, policy_version 73610 (0.0008) [2023-10-12 23:06:39,928][44958] Updated weights for policy 0, policy_version 73250 (0.0011) [2023-10-12 23:06:40,110][44959] Updated weights for policy 1, policy_version 73620 (0.0009) [2023-10-12 23:06:40,300][44958] Updated weights for policy 0, policy_version 73260 (0.0008) [2023-10-12 23:06:40,484][44959] Updated weights for policy 1, policy_version 73630 (0.0007) [2023-10-12 23:06:40,683][44958] Updated weights for policy 0, policy_version 73270 (0.0007) [2023-10-12 23:06:41,063][44958] Updated weights for policy 0, policy_version 73280 (0.0008) [2023-10-12 23:06:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150437888. Throughput: 0: 1637.6, 1: 1648.4. Samples: 37612528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:06:41,444][43579] Avg episode reward: [(0, '272.380'), (1, '284.980')] [2023-10-12 23:06:44,699][44959] Updated weights for policy 1, policy_version 73640 (0.0007) [2023-10-12 23:06:45,070][44959] Updated weights for policy 1, policy_version 73650 (0.0008) [2023-10-12 23:06:45,411][44958] Updated weights for policy 0, policy_version 73290 (0.0009) [2023-10-12 23:06:45,443][44959] Updated weights for policy 1, policy_version 73660 (0.0007) [2023-10-12 23:06:45,792][44958] Updated weights for policy 0, policy_version 73300 (0.0009) [2023-10-12 23:06:46,154][44958] Updated weights for policy 0, policy_version 73310 (0.0008) [2023-10-12 23:06:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150503424. Throughput: 0: 1643.0, 1: 1647.8. Samples: 37631452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:06:46,443][43579] Avg episode reward: [(0, '274.280'), (1, '282.750')] [2023-10-12 23:06:49,441][44959] Updated weights for policy 1, policy_version 73670 (0.0009) [2023-10-12 23:06:49,822][44959] Updated weights for policy 1, policy_version 73680 (0.0010) [2023-10-12 23:06:50,164][44958] Updated weights for policy 0, policy_version 73320 (0.0008) [2023-10-12 23:06:50,189][44959] Updated weights for policy 1, policy_version 73690 (0.0008) [2023-10-12 23:06:50,535][44958] Updated weights for policy 0, policy_version 73330 (0.0009) [2023-10-12 23:06:50,904][44958] Updated weights for policy 0, policy_version 73340 (0.0007) [2023-10-12 23:06:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150568960. Throughput: 0: 1637.6, 1: 1650.3. Samples: 37642756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:06:51,444][43579] Avg episode reward: [(0, '277.260'), (1, '284.790')] [2023-10-12 23:06:54,476][44959] Updated weights for policy 1, policy_version 73700 (0.0007) [2023-10-12 23:06:54,843][44959] Updated weights for policy 1, policy_version 73710 (0.0008) [2023-10-12 23:06:55,213][44959] Updated weights for policy 1, policy_version 73720 (0.0010) [2023-10-12 23:06:55,245][44958] Updated weights for policy 0, policy_version 73350 (0.0007) [2023-10-12 23:06:55,610][44958] Updated weights for policy 0, policy_version 73360 (0.0007) [2023-10-12 23:06:55,982][44958] Updated weights for policy 0, policy_version 73370 (0.0008) [2023-10-12 23:06:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150634496. Throughput: 0: 1636.2, 1: 1641.1. Samples: 37662010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:06:56,443][43579] Avg episode reward: [(0, '280.710'), (1, '284.140')] [2023-10-12 23:06:59,483][44959] Updated weights for policy 1, policy_version 73730 (0.0007) [2023-10-12 23:06:59,847][44959] Updated weights for policy 1, policy_version 73740 (0.0008) [2023-10-12 23:06:59,955][44958] Updated weights for policy 0, policy_version 73380 (0.0008) [2023-10-12 23:07:00,228][44959] Updated weights for policy 1, policy_version 73750 (0.0009) [2023-10-12 23:07:00,321][44958] Updated weights for policy 0, policy_version 73390 (0.0007) [2023-10-12 23:07:00,591][44959] Updated weights for policy 1, policy_version 73760 (0.0008) [2023-10-12 23:07:00,691][44958] Updated weights for policy 0, policy_version 73400 (0.0009) [2023-10-12 23:07:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150700032. Throughput: 0: 1636.8, 1: 1642.1. Samples: 37680432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:01,444][43579] Avg episode reward: [(0, '280.600'), (1, '283.560')] [2023-10-12 23:07:04,673][44959] Updated weights for policy 1, policy_version 73770 (0.0009) [2023-10-12 23:07:04,766][44958] Updated weights for policy 0, policy_version 73410 (0.0008) [2023-10-12 23:07:05,040][44959] Updated weights for policy 1, policy_version 73780 (0.0009) [2023-10-12 23:07:05,141][44958] Updated weights for policy 0, policy_version 73420 (0.0007) [2023-10-12 23:07:05,398][44959] Updated weights for policy 1, policy_version 73790 (0.0008) [2023-10-12 23:07:05,506][44958] Updated weights for policy 0, policy_version 73430 (0.0008) [2023-10-12 23:07:05,878][44958] Updated weights for policy 0, policy_version 73440 (0.0010) [2023-10-12 23:07:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150765568. Throughput: 0: 1638.9, 1: 1641.1. Samples: 37691894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:06,444][43579] Avg episode reward: [(0, '283.210'), (1, '280.750')] [2023-10-12 23:07:09,426][44959] Updated weights for policy 1, policy_version 73800 (0.0009) [2023-10-12 23:07:09,793][44959] Updated weights for policy 1, policy_version 73810 (0.0011) [2023-10-12 23:07:10,164][44959] Updated weights for policy 1, policy_version 73820 (0.0010) [2023-10-12 23:07:10,295][44958] Updated weights for policy 0, policy_version 73450 (0.0009) [2023-10-12 23:07:10,669][44958] Updated weights for policy 0, policy_version 73460 (0.0008) [2023-10-12 23:07:11,040][44958] Updated weights for policy 0, policy_version 73470 (0.0009) [2023-10-12 23:07:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150831104. Throughput: 0: 1639.8, 1: 1635.1. Samples: 37710788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:11,443][43579] Avg episode reward: [(0, '284.590'), (1, '281.440')] [2023-10-12 23:07:14,533][44959] Updated weights for policy 1, policy_version 73830 (0.0009) [2023-10-12 23:07:14,906][44959] Updated weights for policy 1, policy_version 73840 (0.0009) [2023-10-12 23:07:15,004][44958] Updated weights for policy 0, policy_version 73480 (0.0008) [2023-10-12 23:07:15,278][44959] Updated weights for policy 1, policy_version 73850 (0.0008) [2023-10-12 23:07:15,380][44958] Updated weights for policy 0, policy_version 73490 (0.0010) [2023-10-12 23:07:15,752][44958] Updated weights for policy 0, policy_version 73500 (0.0010) [2023-10-12 23:07:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150896640. Throughput: 0: 1641.1, 1: 1639.5. Samples: 37729450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:16,443][43579] Avg episode reward: [(0, '284.790'), (1, '282.810')] [2023-10-12 23:07:19,339][44959] Updated weights for policy 1, policy_version 73860 (0.0008) [2023-10-12 23:07:19,702][44959] Updated weights for policy 1, policy_version 73870 (0.0007) [2023-10-12 23:07:20,044][44958] Updated weights for policy 0, policy_version 73510 (0.0009) [2023-10-12 23:07:20,067][44959] Updated weights for policy 1, policy_version 73880 (0.0007) [2023-10-12 23:07:20,412][44958] Updated weights for policy 0, policy_version 73520 (0.0009) [2023-10-12 23:07:20,785][44958] Updated weights for policy 0, policy_version 73530 (0.0009) [2023-10-12 23:07:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 150962176. Throughput: 0: 1634.2, 1: 1636.8. Samples: 37740560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:21,443][43579] Avg episode reward: [(0, '281.950'), (1, '282.540')] [2023-10-12 23:07:24,351][44959] Updated weights for policy 1, policy_version 73890 (0.0008) [2023-10-12 23:07:24,716][44959] Updated weights for policy 1, policy_version 73900 (0.0009) [2023-10-12 23:07:25,074][44958] Updated weights for policy 0, policy_version 73540 (0.0009) [2023-10-12 23:07:25,092][44959] Updated weights for policy 1, policy_version 73910 (0.0008) [2023-10-12 23:07:25,461][44958] Updated weights for policy 0, policy_version 73550 (0.0009) [2023-10-12 23:07:25,467][44959] Updated weights for policy 1, policy_version 73920 (0.0008) [2023-10-12 23:07:25,842][44958] Updated weights for policy 0, policy_version 73560 (0.0010) [2023-10-12 23:07:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151027712. Throughput: 0: 1634.4, 1: 1633.0. Samples: 37759564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:26,444][43579] Avg episode reward: [(0, '279.360'), (1, '280.830')] [2023-10-12 23:07:29,697][44959] Updated weights for policy 1, policy_version 73930 (0.0008) [2023-10-12 23:07:30,068][44959] Updated weights for policy 1, policy_version 73940 (0.0009) [2023-10-12 23:07:30,083][44958] Updated weights for policy 0, policy_version 73570 (0.0007) [2023-10-12 23:07:30,444][44959] Updated weights for policy 1, policy_version 73950 (0.0008) [2023-10-12 23:07:30,454][44958] Updated weights for policy 0, policy_version 73580 (0.0009) [2023-10-12 23:07:30,817][44958] Updated weights for policy 0, policy_version 73590 (0.0009) [2023-10-12 23:07:31,190][44958] Updated weights for policy 0, policy_version 73600 (0.0007) [2023-10-12 23:07:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151093248. Throughput: 0: 1626.0, 1: 1635.2. Samples: 37778206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:31,443][43579] Avg episode reward: [(0, '274.730'), (1, '283.010')] [2023-10-12 23:07:34,398][44959] Updated weights for policy 1, policy_version 73960 (0.0008) [2023-10-12 23:07:34,763][44959] Updated weights for policy 1, policy_version 73970 (0.0009) [2023-10-12 23:07:35,132][44959] Updated weights for policy 1, policy_version 73980 (0.0009) [2023-10-12 23:07:35,524][44958] Updated weights for policy 0, policy_version 73610 (0.0008) [2023-10-12 23:07:35,899][44958] Updated weights for policy 0, policy_version 73620 (0.0007) [2023-10-12 23:07:36,268][44958] Updated weights for policy 0, policy_version 73630 (0.0007) [2023-10-12 23:07:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151158784. Throughput: 0: 1625.4, 1: 1638.0. Samples: 37789608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:36,443][43579] Avg episode reward: [(0, '273.190'), (1, '285.500')] [2023-10-12 23:07:39,409][44959] Updated weights for policy 1, policy_version 73990 (0.0008) [2023-10-12 23:07:39,781][44959] Updated weights for policy 1, policy_version 74000 (0.0007) [2023-10-12 23:07:40,139][44959] Updated weights for policy 1, policy_version 74010 (0.0008) [2023-10-12 23:07:40,360][44958] Updated weights for policy 0, policy_version 73640 (0.0009) [2023-10-12 23:07:40,733][44958] Updated weights for policy 0, policy_version 73650 (0.0011) [2023-10-12 23:07:41,103][44958] Updated weights for policy 0, policy_version 73660 (0.0008) [2023-10-12 23:07:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151224320. Throughput: 0: 1629.0, 1: 1632.3. Samples: 37808768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:41,444][43579] Avg episode reward: [(0, '272.290'), (1, '280.900')] [2023-10-12 23:07:44,358][44959] Updated weights for policy 1, policy_version 74020 (0.0009) [2023-10-12 23:07:44,733][44959] Updated weights for policy 1, policy_version 74030 (0.0009) [2023-10-12 23:07:45,056][44958] Updated weights for policy 0, policy_version 73670 (0.0009) [2023-10-12 23:07:45,095][44959] Updated weights for policy 1, policy_version 74040 (0.0007) [2023-10-12 23:07:45,438][44958] Updated weights for policy 0, policy_version 73680 (0.0010) [2023-10-12 23:07:45,810][44958] Updated weights for policy 0, policy_version 73690 (0.0008) [2023-10-12 23:07:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151289856. Throughput: 0: 1630.2, 1: 1638.6. Samples: 37827526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:46,443][43579] Avg episode reward: [(0, '271.290'), (1, '283.270')] [2023-10-12 23:07:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000074048_75825152.pth... [2023-10-12 23:07:46,453][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000073696_75464704.pth... [2023-10-12 23:07:46,489][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000072160_73891840.pth [2023-10-12 23:07:46,496][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000072512_74252288.pth [2023-10-12 23:07:49,423][44959] Updated weights for policy 1, policy_version 74050 (0.0007) [2023-10-12 23:07:49,847][44959] Updated weights for policy 1, policy_version 74060 (0.0008) [2023-10-12 23:07:50,078][44958] Updated weights for policy 0, policy_version 73700 (0.0010) [2023-10-12 23:07:50,220][44959] Updated weights for policy 1, policy_version 74070 (0.0008) [2023-10-12 23:07:50,447][44958] Updated weights for policy 0, policy_version 73710 (0.0010) [2023-10-12 23:07:50,590][44959] Updated weights for policy 1, policy_version 74080 (0.0009) [2023-10-12 23:07:50,810][44958] Updated weights for policy 0, policy_version 73720 (0.0007) [2023-10-12 23:07:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151355392. Throughput: 0: 1625.5, 1: 1635.8. Samples: 37838654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:07:51,443][43579] Avg episode reward: [(0, '275.620'), (1, '284.180')] [2023-10-12 23:07:54,624][44959] Updated weights for policy 1, policy_version 74090 (0.0009) [2023-10-12 23:07:54,810][44958] Updated weights for policy 0, policy_version 73730 (0.0008) [2023-10-12 23:07:54,998][44959] Updated weights for policy 1, policy_version 74100 (0.0008) [2023-10-12 23:07:55,183][44958] Updated weights for policy 0, policy_version 73740 (0.0007) [2023-10-12 23:07:55,368][44959] Updated weights for policy 1, policy_version 74110 (0.0008) [2023-10-12 23:07:55,554][44958] Updated weights for policy 0, policy_version 73750 (0.0007) [2023-10-12 23:07:55,921][44958] Updated weights for policy 0, policy_version 73760 (0.0008) [2023-10-12 23:07:56,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151420928. Throughput: 0: 1636.5, 1: 1636.7. Samples: 37858084. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:07:56,444][43579] Avg episode reward: [(0, '274.950'), (1, '281.950')] [2023-10-12 23:07:59,635][44959] Updated weights for policy 1, policy_version 74120 (0.0010) [2023-10-12 23:07:59,998][44959] Updated weights for policy 1, policy_version 74130 (0.0008) [2023-10-12 23:08:00,010][44958] Updated weights for policy 0, policy_version 73770 (0.0008) [2023-10-12 23:08:00,367][44959] Updated weights for policy 1, policy_version 74140 (0.0007) [2023-10-12 23:08:00,386][44958] Updated weights for policy 0, policy_version 73780 (0.0008) [2023-10-12 23:08:00,751][44958] Updated weights for policy 0, policy_version 73790 (0.0009) [2023-10-12 23:08:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151486464. Throughput: 0: 1647.8, 1: 1638.9. Samples: 37877354. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:08:01,443][43579] Avg episode reward: [(0, '274.840'), (1, '278.710')] [2023-10-12 23:08:04,465][44959] Updated weights for policy 1, policy_version 74150 (0.0008) [2023-10-12 23:08:04,837][44959] Updated weights for policy 1, policy_version 74160 (0.0007) [2023-10-12 23:08:04,963][44958] Updated weights for policy 0, policy_version 73800 (0.0010) [2023-10-12 23:08:05,208][44959] Updated weights for policy 1, policy_version 74170 (0.0007) [2023-10-12 23:08:05,334][44958] Updated weights for policy 0, policy_version 73810 (0.0007) [2023-10-12 23:08:05,703][44958] Updated weights for policy 0, policy_version 73820 (0.0008) [2023-10-12 23:08:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151552000. Throughput: 0: 1651.4, 1: 1641.1. Samples: 37888724. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:08:06,443][43579] Avg episode reward: [(0, '277.230'), (1, '280.470')] [2023-10-12 23:08:09,379][44959] Updated weights for policy 1, policy_version 74180 (0.0010) [2023-10-12 23:08:09,747][44959] Updated weights for policy 1, policy_version 74190 (0.0008) [2023-10-12 23:08:10,102][44958] Updated weights for policy 0, policy_version 73830 (0.0008) [2023-10-12 23:08:10,120][44959] Updated weights for policy 1, policy_version 74200 (0.0008) [2023-10-12 23:08:10,492][44958] Updated weights for policy 0, policy_version 73840 (0.0008) [2023-10-12 23:08:10,870][44958] Updated weights for policy 0, policy_version 73850 (0.0008) [2023-10-12 23:08:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151617536. Throughput: 0: 1650.8, 1: 1638.4. Samples: 37907578. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:08:11,443][43579] Avg episode reward: [(0, '278.740'), (1, '285.200')] [2023-10-12 23:08:14,389][44959] Updated weights for policy 1, policy_version 74210 (0.0008) [2023-10-12 23:08:14,765][44959] Updated weights for policy 1, policy_version 74220 (0.0007) [2023-10-12 23:08:15,020][44958] Updated weights for policy 0, policy_version 73860 (0.0008) [2023-10-12 23:08:15,125][44959] Updated weights for policy 1, policy_version 74230 (0.0008) [2023-10-12 23:08:15,384][44958] Updated weights for policy 0, policy_version 73870 (0.0009) [2023-10-12 23:08:15,497][44959] Updated weights for policy 1, policy_version 74240 (0.0007) [2023-10-12 23:08:15,753][44958] Updated weights for policy 0, policy_version 73880 (0.0009) [2023-10-12 23:08:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151683072. Throughput: 0: 1646.0, 1: 1642.3. Samples: 37926178. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:08:16,443][43579] Avg episode reward: [(0, '280.860'), (1, '286.030')] [2023-10-12 23:08:19,643][44959] Updated weights for policy 1, policy_version 74250 (0.0010) [2023-10-12 23:08:20,010][44959] Updated weights for policy 1, policy_version 74260 (0.0008) [2023-10-12 23:08:20,112][44958] Updated weights for policy 0, policy_version 73890 (0.0009) [2023-10-12 23:08:20,372][44959] Updated weights for policy 1, policy_version 74270 (0.0007) [2023-10-12 23:08:20,472][44958] Updated weights for policy 0, policy_version 73900 (0.0009) [2023-10-12 23:08:20,856][44958] Updated weights for policy 0, policy_version 73910 (0.0010) [2023-10-12 23:08:21,225][44958] Updated weights for policy 0, policy_version 73920 (0.0007) [2023-10-12 23:08:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151748608. Throughput: 0: 1646.3, 1: 1637.4. Samples: 37937374. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:08:21,444][43579] Avg episode reward: [(0, '281.470'), (1, '285.180')] [2023-10-12 23:08:24,668][44959] Updated weights for policy 1, policy_version 74280 (0.0010) [2023-10-12 23:08:25,038][44959] Updated weights for policy 1, policy_version 74290 (0.0007) [2023-10-12 23:08:25,268][44958] Updated weights for policy 0, policy_version 73930 (0.0009) [2023-10-12 23:08:25,422][44959] Updated weights for policy 1, policy_version 74300 (0.0007) [2023-10-12 23:08:25,642][44958] Updated weights for policy 0, policy_version 73940 (0.0009) [2023-10-12 23:08:26,018][44958] Updated weights for policy 0, policy_version 73950 (0.0010) [2023-10-12 23:08:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151814144. Throughput: 0: 1640.9, 1: 1643.3. Samples: 37956558. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:08:26,443][43579] Avg episode reward: [(0, '281.590'), (1, '288.140')] [2023-10-12 23:08:29,538][44959] Updated weights for policy 1, policy_version 74310 (0.0008) [2023-10-12 23:08:29,899][44959] Updated weights for policy 1, policy_version 74320 (0.0009) [2023-10-12 23:08:30,134][44958] Updated weights for policy 0, policy_version 73960 (0.0010) [2023-10-12 23:08:30,277][44959] Updated weights for policy 1, policy_version 74330 (0.0009) [2023-10-12 23:08:30,509][44958] Updated weights for policy 0, policy_version 73970 (0.0007) [2023-10-12 23:08:30,885][44958] Updated weights for policy 0, policy_version 73980 (0.0007) [2023-10-12 23:08:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 151879680. Throughput: 0: 1646.8, 1: 1635.2. Samples: 37975218. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:08:31,444][43579] Avg episode reward: [(0, '281.080'), (1, '288.650')] [2023-10-12 23:08:34,512][44959] Updated weights for policy 1, policy_version 74340 (0.0010) [2023-10-12 23:08:34,914][44959] Updated weights for policy 1, policy_version 74350 (0.0010) [2023-10-12 23:08:35,136][44958] Updated weights for policy 0, policy_version 73990 (0.0007) [2023-10-12 23:08:35,280][44959] Updated weights for policy 1, policy_version 74360 (0.0008) [2023-10-12 23:08:35,505][44958] Updated weights for policy 0, policy_version 74000 (0.0007) [2023-10-12 23:08:35,867][44958] Updated weights for policy 0, policy_version 74010 (0.0007) [2023-10-12 23:08:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 151945216. Throughput: 0: 1647.4, 1: 1642.4. Samples: 37986698. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:08:36,444][43579] Avg episode reward: [(0, '275.880'), (1, '285.940')] [2023-10-12 23:08:39,571][44959] Updated weights for policy 1, policy_version 74370 (0.0008) [2023-10-12 23:08:39,931][44958] Updated weights for policy 0, policy_version 74020 (0.0007) [2023-10-12 23:08:39,941][44959] Updated weights for policy 1, policy_version 74380 (0.0009) [2023-10-12 23:08:40,304][44958] Updated weights for policy 0, policy_version 74030 (0.0008) [2023-10-12 23:08:40,309][44959] Updated weights for policy 1, policy_version 74390 (0.0008) [2023-10-12 23:08:40,676][44958] Updated weights for policy 0, policy_version 74040 (0.0007) [2023-10-12 23:08:40,684][44959] Updated weights for policy 1, policy_version 74400 (0.0007) [2023-10-12 23:08:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152010752. Throughput: 0: 1639.4, 1: 1640.8. Samples: 38005694. Policy #0 lag: (min: 9.0, avg: 13.3, max: 41.0) [2023-10-12 23:08:41,444][43579] Avg episode reward: [(0, '269.830'), (1, '283.920')] [2023-10-12 23:08:44,691][44959] Updated weights for policy 1, policy_version 74410 (0.0010) [2023-10-12 23:08:44,878][44958] Updated weights for policy 0, policy_version 74050 (0.0009) [2023-10-12 23:08:45,067][44959] Updated weights for policy 1, policy_version 74420 (0.0007) [2023-10-12 23:08:45,262][44958] Updated weights for policy 0, policy_version 74060 (0.0009) [2023-10-12 23:08:45,436][44959] Updated weights for policy 1, policy_version 74430 (0.0007) [2023-10-12 23:08:45,634][44958] Updated weights for policy 0, policy_version 74070 (0.0008) [2023-10-12 23:08:46,013][44958] Updated weights for policy 0, policy_version 74080 (0.0008) [2023-10-12 23:08:46,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152076288. Throughput: 0: 1628.7, 1: 1635.7. Samples: 38024252. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:08:46,443][43579] Avg episode reward: [(0, '269.500'), (1, '284.360')] [2023-10-12 23:08:49,756][44959] Updated weights for policy 1, policy_version 74440 (0.0008) [2023-10-12 23:08:50,121][44959] Updated weights for policy 1, policy_version 74450 (0.0007) [2023-10-12 23:08:50,127][44958] Updated weights for policy 0, policy_version 74090 (0.0009) [2023-10-12 23:08:50,492][44959] Updated weights for policy 1, policy_version 74460 (0.0007) [2023-10-12 23:08:50,507][44958] Updated weights for policy 0, policy_version 74100 (0.0007) [2023-10-12 23:08:50,868][44958] Updated weights for policy 0, policy_version 74110 (0.0007) [2023-10-12 23:08:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152141824. Throughput: 0: 1628.6, 1: 1631.7. Samples: 38035438. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:08:51,443][43579] Avg episode reward: [(0, '266.280'), (1, '285.720')] [2023-10-12 23:08:54,772][44959] Updated weights for policy 1, policy_version 74470 (0.0009) [2023-10-12 23:08:55,143][44959] Updated weights for policy 1, policy_version 74480 (0.0008) [2023-10-12 23:08:55,261][44958] Updated weights for policy 0, policy_version 74120 (0.0007) [2023-10-12 23:08:55,497][44959] Updated weights for policy 1, policy_version 74490 (0.0009) [2023-10-12 23:08:55,635][44958] Updated weights for policy 0, policy_version 74130 (0.0007) [2023-10-12 23:08:56,000][44958] Updated weights for policy 0, policy_version 74140 (0.0007) [2023-10-12 23:08:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152207360. Throughput: 0: 1630.7, 1: 1640.3. Samples: 38054770. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:08:56,444][43579] Avg episode reward: [(0, '265.610'), (1, '286.390')] [2023-10-12 23:08:59,683][44959] Updated weights for policy 1, policy_version 74500 (0.0008) [2023-10-12 23:09:00,049][44959] Updated weights for policy 1, policy_version 74510 (0.0010) [2023-10-12 23:09:00,082][44958] Updated weights for policy 0, policy_version 74150 (0.0009) [2023-10-12 23:09:00,411][44959] Updated weights for policy 1, policy_version 74520 (0.0007) [2023-10-12 23:09:00,440][44958] Updated weights for policy 0, policy_version 74160 (0.0009) [2023-10-12 23:09:00,810][44958] Updated weights for policy 0, policy_version 74170 (0.0010) [2023-10-12 23:09:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152272896. Throughput: 0: 1637.9, 1: 1635.9. Samples: 38073498. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:09:01,444][43579] Avg episode reward: [(0, '270.350'), (1, '286.830')] [2023-10-12 23:09:04,363][44959] Updated weights for policy 1, policy_version 74530 (0.0007) [2023-10-12 23:09:04,736][44959] Updated weights for policy 1, policy_version 74540 (0.0008) [2023-10-12 23:09:05,096][44959] Updated weights for policy 1, policy_version 74550 (0.0008) [2023-10-12 23:09:05,131][44958] Updated weights for policy 0, policy_version 74180 (0.0010) [2023-10-12 23:09:05,461][44959] Updated weights for policy 1, policy_version 74560 (0.0008) [2023-10-12 23:09:05,493][44958] Updated weights for policy 0, policy_version 74190 (0.0007) [2023-10-12 23:09:05,878][44958] Updated weights for policy 0, policy_version 74200 (0.0009) [2023-10-12 23:09:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152338432. Throughput: 0: 1637.9, 1: 1638.0. Samples: 38084792. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:09:06,444][43579] Avg episode reward: [(0, '272.260'), (1, '289.280')] [2023-10-12 23:09:09,619][44959] Updated weights for policy 1, policy_version 74570 (0.0008) [2023-10-12 23:09:09,978][44959] Updated weights for policy 1, policy_version 74580 (0.0009) [2023-10-12 23:09:10,103][44958] Updated weights for policy 0, policy_version 74210 (0.0010) [2023-10-12 23:09:10,357][44959] Updated weights for policy 1, policy_version 74590 (0.0008) [2023-10-12 23:09:10,476][44958] Updated weights for policy 0, policy_version 74220 (0.0009) [2023-10-12 23:09:10,860][44958] Updated weights for policy 0, policy_version 74230 (0.0010) [2023-10-12 23:09:11,227][44958] Updated weights for policy 0, policy_version 74240 (0.0010) [2023-10-12 23:09:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152403968. Throughput: 0: 1640.3, 1: 1637.8. Samples: 38104074. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:09:11,443][43579] Avg episode reward: [(0, '272.830'), (1, '293.600')] [2023-10-12 23:09:11,444][44583] Saving new best policy, reward=293.600! [2023-10-12 23:09:14,620][44959] Updated weights for policy 1, policy_version 74600 (0.0009) [2023-10-12 23:09:14,994][44959] Updated weights for policy 1, policy_version 74610 (0.0007) [2023-10-12 23:09:15,329][44958] Updated weights for policy 0, policy_version 74250 (0.0007) [2023-10-12 23:09:15,353][44959] Updated weights for policy 1, policy_version 74620 (0.0010) [2023-10-12 23:09:15,699][44958] Updated weights for policy 0, policy_version 74260 (0.0009) [2023-10-12 23:09:16,076][44958] Updated weights for policy 0, policy_version 74270 (0.0008) [2023-10-12 23:09:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 152469504. Throughput: 0: 1638.6, 1: 1641.5. Samples: 38122820. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:09:16,444][43579] Avg episode reward: [(0, '269.780'), (1, '292.320')] [2023-10-12 23:09:19,471][44959] Updated weights for policy 1, policy_version 74630 (0.0009) [2023-10-12 23:09:19,854][44959] Updated weights for policy 1, policy_version 74640 (0.0007) [2023-10-12 23:09:20,215][44959] Updated weights for policy 1, policy_version 74650 (0.0007) [2023-10-12 23:09:20,263][44958] Updated weights for policy 0, policy_version 74280 (0.0009) [2023-10-12 23:09:20,638][44958] Updated weights for policy 0, policy_version 74290 (0.0009) [2023-10-12 23:09:21,018][44958] Updated weights for policy 0, policy_version 74300 (0.0007) [2023-10-12 23:09:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152535040. Throughput: 0: 1635.8, 1: 1639.7. Samples: 38134096. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:09:21,443][43579] Avg episode reward: [(0, '277.080'), (1, '287.090')] [2023-10-12 23:09:24,423][44959] Updated weights for policy 1, policy_version 74660 (0.0008) [2023-10-12 23:09:24,791][44959] Updated weights for policy 1, policy_version 74670 (0.0008) [2023-10-12 23:09:25,131][44958] Updated weights for policy 0, policy_version 74310 (0.0007) [2023-10-12 23:09:25,161][44959] Updated weights for policy 1, policy_version 74680 (0.0009) [2023-10-12 23:09:25,510][44958] Updated weights for policy 0, policy_version 74320 (0.0009) [2023-10-12 23:09:25,882][44958] Updated weights for policy 0, policy_version 74330 (0.0010) [2023-10-12 23:09:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152600576. Throughput: 0: 1639.1, 1: 1636.2. Samples: 38153082. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:09:26,444][43579] Avg episode reward: [(0, '274.040'), (1, '288.110')] [2023-10-12 23:09:29,476][44959] Updated weights for policy 1, policy_version 74690 (0.0009) [2023-10-12 23:09:29,843][44959] Updated weights for policy 1, policy_version 74700 (0.0008) [2023-10-12 23:09:30,084][44958] Updated weights for policy 0, policy_version 74340 (0.0009) [2023-10-12 23:09:30,203][44959] Updated weights for policy 1, policy_version 74710 (0.0009) [2023-10-12 23:09:30,454][44958] Updated weights for policy 0, policy_version 74350 (0.0007) [2023-10-12 23:09:30,571][44959] Updated weights for policy 1, policy_version 74720 (0.0008) [2023-10-12 23:09:30,822][44958] Updated weights for policy 0, policy_version 74360 (0.0008) [2023-10-12 23:09:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 152666112. Throughput: 0: 1642.2, 1: 1636.0. Samples: 38171772. Policy #0 lag: (min: 22.0, avg: 22.8, max: 42.0) [2023-10-12 23:09:31,443][43579] Avg episode reward: [(0, '274.280'), (1, '287.820')] [2023-10-12 23:09:34,740][44959] Updated weights for policy 1, policy_version 74730 (0.0009) [2023-10-12 23:09:35,003][44958] Updated weights for policy 0, policy_version 74370 (0.0007) [2023-10-12 23:09:35,100][44959] Updated weights for policy 1, policy_version 74740 (0.0007) [2023-10-12 23:09:35,368][44958] Updated weights for policy 0, policy_version 74380 (0.0009) [2023-10-12 23:09:35,460][44959] Updated weights for policy 1, policy_version 74750 (0.0009) [2023-10-12 23:09:35,737][44958] Updated weights for policy 0, policy_version 74390 (0.0009) [2023-10-12 23:09:36,112][44958] Updated weights for policy 0, policy_version 74400 (0.0009) [2023-10-12 23:09:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152731648. Throughput: 0: 1637.6, 1: 1641.2. Samples: 38182984. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:09:36,443][43579] Avg episode reward: [(0, '271.570'), (1, '285.480')] [2023-10-12 23:09:39,677][44959] Updated weights for policy 1, policy_version 74760 (0.0009) [2023-10-12 23:09:40,050][44959] Updated weights for policy 1, policy_version 74770 (0.0007) [2023-10-12 23:09:40,422][44959] Updated weights for policy 1, policy_version 74780 (0.0007) [2023-10-12 23:09:40,515][44958] Updated weights for policy 0, policy_version 74410 (0.0007) [2023-10-12 23:09:40,900][44958] Updated weights for policy 0, policy_version 74420 (0.0010) [2023-10-12 23:09:41,276][44958] Updated weights for policy 0, policy_version 74430 (0.0009) [2023-10-12 23:09:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152797184. Throughput: 0: 1644.7, 1: 1634.9. Samples: 38202354. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:09:41,443][43579] Avg episode reward: [(0, '269.750'), (1, '284.720')] [2023-10-12 23:09:44,800][44959] Updated weights for policy 1, policy_version 74790 (0.0008) [2023-10-12 23:09:45,157][44959] Updated weights for policy 1, policy_version 74800 (0.0008) [2023-10-12 23:09:45,202][44958] Updated weights for policy 0, policy_version 74440 (0.0007) [2023-10-12 23:09:45,532][44959] Updated weights for policy 1, policy_version 74810 (0.0007) [2023-10-12 23:09:45,583][44958] Updated weights for policy 0, policy_version 74450 (0.0008) [2023-10-12 23:09:45,948][44958] Updated weights for policy 0, policy_version 74460 (0.0009) [2023-10-12 23:09:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 152862720. Throughput: 0: 1633.6, 1: 1633.9. Samples: 38220534. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:09:46,444][43579] Avg episode reward: [(0, '270.510'), (1, '285.040')] [2023-10-12 23:09:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000074816_76611584.pth... [2023-10-12 23:09:46,453][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000074464_76251136.pth... [2023-10-12 23:09:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000073280_75038720.pth [2023-10-12 23:09:46,497][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000072928_74678272.pth [2023-10-12 23:09:49,564][44959] Updated weights for policy 1, policy_version 74820 (0.0007) [2023-10-12 23:09:49,938][44959] Updated weights for policy 1, policy_version 74830 (0.0008) [2023-10-12 23:09:50,072][44958] Updated weights for policy 0, policy_version 74470 (0.0008) [2023-10-12 23:09:50,305][44959] Updated weights for policy 1, policy_version 74840 (0.0008) [2023-10-12 23:09:50,441][44958] Updated weights for policy 0, policy_version 74480 (0.0008) [2023-10-12 23:09:50,818][44958] Updated weights for policy 0, policy_version 74490 (0.0008) [2023-10-12 23:09:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152928256. Throughput: 0: 1636.6, 1: 1630.0. Samples: 38231790. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:09:51,443][43579] Avg episode reward: [(0, '266.580'), (1, '287.840')] [2023-10-12 23:09:54,391][44959] Updated weights for policy 1, policy_version 74850 (0.0007) [2023-10-12 23:09:54,759][44959] Updated weights for policy 1, policy_version 74860 (0.0011) [2023-10-12 23:09:54,944][44958] Updated weights for policy 0, policy_version 74500 (0.0007) [2023-10-12 23:09:55,126][44959] Updated weights for policy 1, policy_version 74870 (0.0008) [2023-10-12 23:09:55,318][44958] Updated weights for policy 0, policy_version 74510 (0.0009) [2023-10-12 23:09:55,484][44959] Updated weights for policy 1, policy_version 74880 (0.0009) [2023-10-12 23:09:55,691][44958] Updated weights for policy 0, policy_version 74520 (0.0010) [2023-10-12 23:09:56,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 152993792. Throughput: 0: 1635.0, 1: 1632.2. Samples: 38251098. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:09:56,444][43579] Avg episode reward: [(0, '270.100'), (1, '284.780')] [2023-10-12 23:09:59,718][44958] Updated weights for policy 0, policy_version 74530 (0.0009) [2023-10-12 23:09:59,792][44959] Updated weights for policy 1, policy_version 74890 (0.0009) [2023-10-12 23:10:00,085][44958] Updated weights for policy 0, policy_version 74540 (0.0008) [2023-10-12 23:10:00,175][44959] Updated weights for policy 1, policy_version 74900 (0.0008) [2023-10-12 23:10:00,462][44958] Updated weights for policy 0, policy_version 74550 (0.0008) [2023-10-12 23:10:00,540][44959] Updated weights for policy 1, policy_version 74910 (0.0008) [2023-10-12 23:10:00,830][44958] Updated weights for policy 0, policy_version 74560 (0.0007) [2023-10-12 23:10:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153059328. Throughput: 0: 1637.5, 1: 1629.1. Samples: 38269816. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:10:01,444][43579] Avg episode reward: [(0, '272.290'), (1, '285.830')] [2023-10-12 23:10:04,644][44959] Updated weights for policy 1, policy_version 74920 (0.0007) [2023-10-12 23:10:05,019][44959] Updated weights for policy 1, policy_version 74930 (0.0009) [2023-10-12 23:10:05,216][44958] Updated weights for policy 0, policy_version 74570 (0.0008) [2023-10-12 23:10:05,389][44959] Updated weights for policy 1, policy_version 74940 (0.0007) [2023-10-12 23:10:05,592][44958] Updated weights for policy 0, policy_version 74580 (0.0008) [2023-10-12 23:10:05,965][44958] Updated weights for policy 0, policy_version 74590 (0.0011) [2023-10-12 23:10:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153124864. Throughput: 0: 1634.5, 1: 1630.7. Samples: 38281030. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:10:06,444][43579] Avg episode reward: [(0, '280.180'), (1, '289.090')] [2023-10-12 23:10:09,630][44959] Updated weights for policy 1, policy_version 74950 (0.0009) [2023-10-12 23:10:09,989][44959] Updated weights for policy 1, policy_version 74960 (0.0011) [2023-10-12 23:10:10,321][44958] Updated weights for policy 0, policy_version 74600 (0.0009) [2023-10-12 23:10:10,358][44959] Updated weights for policy 1, policy_version 74970 (0.0007) [2023-10-12 23:10:10,703][44958] Updated weights for policy 0, policy_version 74610 (0.0009) [2023-10-12 23:10:11,073][44958] Updated weights for policy 0, policy_version 74620 (0.0009) [2023-10-12 23:10:11,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153190400. Throughput: 0: 1637.0, 1: 1641.4. Samples: 38300608. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:10:11,443][43579] Avg episode reward: [(0, '279.680'), (1, '285.400')] [2023-10-12 23:10:14,553][44959] Updated weights for policy 1, policy_version 74980 (0.0010) [2023-10-12 23:10:14,930][44959] Updated weights for policy 1, policy_version 74990 (0.0009) [2023-10-12 23:10:15,088][44958] Updated weights for policy 0, policy_version 74630 (0.0009) [2023-10-12 23:10:15,301][44959] Updated weights for policy 1, policy_version 75000 (0.0008) [2023-10-12 23:10:15,452][44958] Updated weights for policy 0, policy_version 74640 (0.0008) [2023-10-12 23:10:15,826][44958] Updated weights for policy 0, policy_version 74650 (0.0010) [2023-10-12 23:10:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153255936. Throughput: 0: 1629.6, 1: 1640.5. Samples: 38318928. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:10:16,444][43579] Avg episode reward: [(0, '283.080'), (1, '284.970')] [2023-10-12 23:10:19,658][44959] Updated weights for policy 1, policy_version 75010 (0.0008) [2023-10-12 23:10:20,031][44959] Updated weights for policy 1, policy_version 75020 (0.0007) [2023-10-12 23:10:20,082][44958] Updated weights for policy 0, policy_version 74660 (0.0009) [2023-10-12 23:10:20,394][44959] Updated weights for policy 1, policy_version 75030 (0.0007) [2023-10-12 23:10:20,455][44958] Updated weights for policy 0, policy_version 74670 (0.0007) [2023-10-12 23:10:20,759][44959] Updated weights for policy 1, policy_version 75040 (0.0008) [2023-10-12 23:10:20,832][44958] Updated weights for policy 0, policy_version 74680 (0.0010) [2023-10-12 23:10:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153321472. Throughput: 0: 1632.3, 1: 1638.4. Samples: 38330164. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:10:21,443][43579] Avg episode reward: [(0, '285.120'), (1, '287.070')] [2023-10-12 23:10:24,838][44959] Updated weights for policy 1, policy_version 75050 (0.0009) [2023-10-12 23:10:25,197][44959] Updated weights for policy 1, policy_version 75060 (0.0010) [2023-10-12 23:10:25,220][44958] Updated weights for policy 0, policy_version 74690 (0.0010) [2023-10-12 23:10:25,568][44959] Updated weights for policy 1, policy_version 75070 (0.0007) [2023-10-12 23:10:25,595][44958] Updated weights for policy 0, policy_version 74700 (0.0009) [2023-10-12 23:10:25,972][44958] Updated weights for policy 0, policy_version 74710 (0.0010) [2023-10-12 23:10:26,343][44958] Updated weights for policy 0, policy_version 74720 (0.0010) [2023-10-12 23:10:26,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153387008. Throughput: 0: 1630.4, 1: 1642.9. Samples: 38349654. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:10:26,443][43579] Avg episode reward: [(0, '283.460'), (1, '289.780')] [2023-10-12 23:10:29,901][44959] Updated weights for policy 1, policy_version 75080 (0.0008) [2023-10-12 23:10:30,270][44959] Updated weights for policy 1, policy_version 75090 (0.0010) [2023-10-12 23:10:30,468][44958] Updated weights for policy 0, policy_version 74730 (0.0009) [2023-10-12 23:10:30,639][44959] Updated weights for policy 1, policy_version 75100 (0.0009) [2023-10-12 23:10:30,831][44958] Updated weights for policy 0, policy_version 74740 (0.0009) [2023-10-12 23:10:31,207][44958] Updated weights for policy 0, policy_version 74750 (0.0007) [2023-10-12 23:10:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153452544. Throughput: 0: 1639.4, 1: 1638.0. Samples: 38368014. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:10:31,443][43579] Avg episode reward: [(0, '283.940'), (1, '289.380')] [2023-10-12 23:10:34,793][44959] Updated weights for policy 1, policy_version 75110 (0.0011) [2023-10-12 23:10:35,156][44959] Updated weights for policy 1, policy_version 75120 (0.0008) [2023-10-12 23:10:35,262][44958] Updated weights for policy 0, policy_version 74760 (0.0008) [2023-10-12 23:10:35,534][44959] Updated weights for policy 1, policy_version 75130 (0.0007) [2023-10-12 23:10:35,637][44958] Updated weights for policy 0, policy_version 74770 (0.0007) [2023-10-12 23:10:36,012][44958] Updated weights for policy 0, policy_version 74780 (0.0008) [2023-10-12 23:10:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153518080. Throughput: 0: 1637.8, 1: 1641.2. Samples: 38379346. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:10:36,444][43579] Avg episode reward: [(0, '278.050'), (1, '291.940')] [2023-10-12 23:10:39,536][44959] Updated weights for policy 1, policy_version 75140 (0.0008) [2023-10-12 23:10:39,910][44959] Updated weights for policy 1, policy_version 75150 (0.0008) [2023-10-12 23:10:40,273][44959] Updated weights for policy 1, policy_version 75160 (0.0009) [2023-10-12 23:10:40,322][44958] Updated weights for policy 0, policy_version 74790 (0.0010) [2023-10-12 23:10:40,696][44958] Updated weights for policy 0, policy_version 74800 (0.0009) [2023-10-12 23:10:41,068][44958] Updated weights for policy 0, policy_version 74810 (0.0010) [2023-10-12 23:10:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153583616. Throughput: 0: 1644.0, 1: 1640.8. Samples: 38398912. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:10:41,443][43579] Avg episode reward: [(0, '277.030'), (1, '291.660')] [2023-10-12 23:10:44,466][44959] Updated weights for policy 1, policy_version 75170 (0.0009) [2023-10-12 23:10:44,825][44959] Updated weights for policy 1, policy_version 75180 (0.0010) [2023-10-12 23:10:45,194][44959] Updated weights for policy 1, policy_version 75190 (0.0008) [2023-10-12 23:10:45,200][44958] Updated weights for policy 0, policy_version 74820 (0.0008) [2023-10-12 23:10:45,559][44959] Updated weights for policy 1, policy_version 75200 (0.0008) [2023-10-12 23:10:45,572][44958] Updated weights for policy 0, policy_version 74830 (0.0009) [2023-10-12 23:10:45,947][44958] Updated weights for policy 0, policy_version 74840 (0.0010) [2023-10-12 23:10:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153649152. Throughput: 0: 1640.4, 1: 1638.5. Samples: 38417368. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:10:46,444][43579] Avg episode reward: [(0, '279.510'), (1, '288.060')] [2023-10-12 23:10:49,844][44959] Updated weights for policy 1, policy_version 75210 (0.0009) [2023-10-12 23:10:50,186][44958] Updated weights for policy 0, policy_version 74850 (0.0008) [2023-10-12 23:10:50,217][44959] Updated weights for policy 1, policy_version 75220 (0.0008) [2023-10-12 23:10:50,560][44958] Updated weights for policy 0, policy_version 74860 (0.0007) [2023-10-12 23:10:50,586][44959] Updated weights for policy 1, policy_version 75230 (0.0007) [2023-10-12 23:10:50,929][44958] Updated weights for policy 0, policy_version 74870 (0.0009) [2023-10-12 23:10:51,299][44958] Updated weights for policy 0, policy_version 74880 (0.0009) [2023-10-12 23:10:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153714688. Throughput: 0: 1641.1, 1: 1637.0. Samples: 38428546. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:10:51,443][43579] Avg episode reward: [(0, '279.050'), (1, '287.060')] [2023-10-12 23:10:54,919][44959] Updated weights for policy 1, policy_version 75240 (0.0009) [2023-10-12 23:10:55,287][44959] Updated weights for policy 1, policy_version 75250 (0.0009) [2023-10-12 23:10:55,360][44958] Updated weights for policy 0, policy_version 74890 (0.0008) [2023-10-12 23:10:55,662][44959] Updated weights for policy 1, policy_version 75260 (0.0008) [2023-10-12 23:10:55,736][44958] Updated weights for policy 0, policy_version 74900 (0.0008) [2023-10-12 23:10:56,119][44958] Updated weights for policy 0, policy_version 74910 (0.0009) [2023-10-12 23:10:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153780224. Throughput: 0: 1634.1, 1: 1637.9. Samples: 38447850. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:10:56,444][43579] Avg episode reward: [(0, '263.050'), (1, '286.820')] [2023-10-12 23:10:59,742][44959] Updated weights for policy 1, policy_version 75270 (0.0008) [2023-10-12 23:11:00,121][44959] Updated weights for policy 1, policy_version 75280 (0.0008) [2023-10-12 23:11:00,407][44958] Updated weights for policy 0, policy_version 74920 (0.0010) [2023-10-12 23:11:00,495][44959] Updated weights for policy 1, policy_version 75290 (0.0007) [2023-10-12 23:11:00,790][44958] Updated weights for policy 0, policy_version 74930 (0.0010) [2023-10-12 23:11:01,156][44958] Updated weights for policy 0, policy_version 74940 (0.0008) [2023-10-12 23:11:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153845760. Throughput: 0: 1639.0, 1: 1637.1. Samples: 38466354. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:11:01,443][43579] Avg episode reward: [(0, '263.650'), (1, '288.270')] [2023-10-12 23:11:04,543][44959] Updated weights for policy 1, policy_version 75300 (0.0010) [2023-10-12 23:11:04,914][44959] Updated weights for policy 1, policy_version 75310 (0.0011) [2023-10-12 23:11:05,276][44959] Updated weights for policy 1, policy_version 75320 (0.0009) [2023-10-12 23:11:05,332][44958] Updated weights for policy 0, policy_version 74950 (0.0008) [2023-10-12 23:11:05,707][44958] Updated weights for policy 0, policy_version 74960 (0.0008) [2023-10-12 23:11:06,079][44958] Updated weights for policy 0, policy_version 74970 (0.0007) [2023-10-12 23:11:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 153911296. Throughput: 0: 1634.7, 1: 1637.8. Samples: 38477428. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:11:06,444][43579] Avg episode reward: [(0, '263.930'), (1, '290.880')] [2023-10-12 23:11:09,550][44959] Updated weights for policy 1, policy_version 75330 (0.0008) [2023-10-12 23:11:09,918][44959] Updated weights for policy 1, policy_version 75340 (0.0009) [2023-10-12 23:11:10,277][44959] Updated weights for policy 1, policy_version 75350 (0.0008) [2023-10-12 23:11:10,363][44958] Updated weights for policy 0, policy_version 74980 (0.0008) [2023-10-12 23:11:10,649][44959] Updated weights for policy 1, policy_version 75360 (0.0010) [2023-10-12 23:11:10,751][44958] Updated weights for policy 0, policy_version 74990 (0.0007) [2023-10-12 23:11:11,127][44958] Updated weights for policy 0, policy_version 75000 (0.0008) [2023-10-12 23:11:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 153976832. Throughput: 0: 1639.5, 1: 1635.8. Samples: 38497044. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:11:11,444][43579] Avg episode reward: [(0, '262.210'), (1, '290.560')] [2023-10-12 23:11:14,809][44959] Updated weights for policy 1, policy_version 75370 (0.0009) [2023-10-12 23:11:15,126][44958] Updated weights for policy 0, policy_version 75010 (0.0008) [2023-10-12 23:11:15,177][44959] Updated weights for policy 1, policy_version 75380 (0.0009) [2023-10-12 23:11:15,492][44958] Updated weights for policy 0, policy_version 75020 (0.0008) [2023-10-12 23:11:15,538][44959] Updated weights for policy 1, policy_version 75390 (0.0008) [2023-10-12 23:11:15,874][44958] Updated weights for policy 0, policy_version 75030 (0.0009) [2023-10-12 23:11:16,240][44958] Updated weights for policy 0, policy_version 75040 (0.0008) [2023-10-12 23:11:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154042368. Throughput: 0: 1638.8, 1: 1642.1. Samples: 38515654. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:11:16,443][43579] Avg episode reward: [(0, '261.980'), (1, '293.050')] [2023-10-12 23:11:19,491][44959] Updated weights for policy 1, policy_version 75400 (0.0008) [2023-10-12 23:11:19,865][44959] Updated weights for policy 1, policy_version 75410 (0.0007) [2023-10-12 23:11:20,230][44959] Updated weights for policy 1, policy_version 75420 (0.0007) [2023-10-12 23:11:20,379][44958] Updated weights for policy 0, policy_version 75050 (0.0009) [2023-10-12 23:11:20,758][44958] Updated weights for policy 0, policy_version 75060 (0.0009) [2023-10-12 23:11:21,131][44958] Updated weights for policy 0, policy_version 75070 (0.0007) [2023-10-12 23:11:21,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154107904. Throughput: 0: 1633.9, 1: 1648.8. Samples: 38527064. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:11:21,443][43579] Avg episode reward: [(0, '266.360'), (1, '291.190')] [2023-10-12 23:11:24,128][44959] Updated weights for policy 1, policy_version 75430 (0.0007) [2023-10-12 23:11:24,495][44959] Updated weights for policy 1, policy_version 75440 (0.0008) [2023-10-12 23:11:24,873][44959] Updated weights for policy 1, policy_version 75450 (0.0008) [2023-10-12 23:11:25,457][44958] Updated weights for policy 0, policy_version 75080 (0.0008) [2023-10-12 23:11:25,824][44958] Updated weights for policy 0, policy_version 75090 (0.0011) [2023-10-12 23:11:26,201][44958] Updated weights for policy 0, policy_version 75100 (0.0010) [2023-10-12 23:11:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154173440. Throughput: 0: 1632.0, 1: 1642.4. Samples: 38546262. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:11:26,443][43579] Avg episode reward: [(0, '273.430'), (1, '288.840')] [2023-10-12 23:11:29,096][44959] Updated weights for policy 1, policy_version 75460 (0.0010) [2023-10-12 23:11:29,458][44959] Updated weights for policy 1, policy_version 75470 (0.0007) [2023-10-12 23:11:29,823][44959] Updated weights for policy 1, policy_version 75480 (0.0008) [2023-10-12 23:11:30,157][44958] Updated weights for policy 0, policy_version 75110 (0.0008) [2023-10-12 23:11:30,527][44958] Updated weights for policy 0, policy_version 75120 (0.0009) [2023-10-12 23:11:30,900][44958] Updated weights for policy 0, policy_version 75130 (0.0009) [2023-10-12 23:11:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154238976. Throughput: 0: 1633.8, 1: 1660.8. Samples: 38565626. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:11:31,444][43579] Avg episode reward: [(0, '272.690'), (1, '286.330')] [2023-10-12 23:11:34,106][44959] Updated weights for policy 1, policy_version 75490 (0.0009) [2023-10-12 23:11:34,520][44959] Updated weights for policy 1, policy_version 75500 (0.0009) [2023-10-12 23:11:34,883][44959] Updated weights for policy 1, policy_version 75510 (0.0008) [2023-10-12 23:11:34,971][44958] Updated weights for policy 0, policy_version 75140 (0.0008) [2023-10-12 23:11:35,252][44959] Updated weights for policy 1, policy_version 75520 (0.0008) [2023-10-12 23:11:35,341][44958] Updated weights for policy 0, policy_version 75150 (0.0007) [2023-10-12 23:11:35,708][44958] Updated weights for policy 0, policy_version 75160 (0.0009) [2023-10-12 23:11:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154304512. Throughput: 0: 1639.7, 1: 1657.2. Samples: 38576906. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:11:36,443][43579] Avg episode reward: [(0, '272.700'), (1, '287.810')] [2023-10-12 23:11:39,288][44959] Updated weights for policy 1, policy_version 75530 (0.0009) [2023-10-12 23:11:39,655][44959] Updated weights for policy 1, policy_version 75540 (0.0008) [2023-10-12 23:11:40,022][44959] Updated weights for policy 1, policy_version 75550 (0.0009) [2023-10-12 23:11:40,044][44958] Updated weights for policy 0, policy_version 75170 (0.0008) [2023-10-12 23:11:40,423][44958] Updated weights for policy 0, policy_version 75180 (0.0009) [2023-10-12 23:11:40,803][44958] Updated weights for policy 0, policy_version 75190 (0.0008) [2023-10-12 23:11:41,165][44958] Updated weights for policy 0, policy_version 75200 (0.0009) [2023-10-12 23:11:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154370048. Throughput: 0: 1645.7, 1: 1637.6. Samples: 38595598. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:11:41,444][43579] Avg episode reward: [(0, '280.800'), (1, '286.920')] [2023-10-12 23:11:44,258][44959] Updated weights for policy 1, policy_version 75560 (0.0008) [2023-10-12 23:11:44,633][44959] Updated weights for policy 1, policy_version 75570 (0.0009) [2023-10-12 23:11:45,003][44959] Updated weights for policy 1, policy_version 75580 (0.0008) [2023-10-12 23:11:45,235][44958] Updated weights for policy 0, policy_version 75210 (0.0010) [2023-10-12 23:11:45,617][44958] Updated weights for policy 0, policy_version 75220 (0.0008) [2023-10-12 23:11:45,979][44958] Updated weights for policy 0, policy_version 75230 (0.0009) [2023-10-12 23:11:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154435584. Throughput: 0: 1646.1, 1: 1652.8. Samples: 38614806. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:11:46,444][43579] Avg episode reward: [(0, '283.660'), (1, '287.060')] [2023-10-12 23:11:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000075584_77398016.pth... [2023-10-12 23:11:46,456][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000075232_77037568.pth... [2023-10-12 23:11:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000073696_75464704.pth [2023-10-12 23:11:46,497][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000074048_75825152.pth [2023-10-12 23:11:49,171][44959] Updated weights for policy 1, policy_version 75590 (0.0008) [2023-10-12 23:11:49,533][44959] Updated weights for policy 1, policy_version 75600 (0.0010) [2023-10-12 23:11:49,910][44959] Updated weights for policy 1, policy_version 75610 (0.0008) [2023-10-12 23:11:50,309][44958] Updated weights for policy 0, policy_version 75240 (0.0007) [2023-10-12 23:11:50,684][44958] Updated weights for policy 0, policy_version 75250 (0.0008) [2023-10-12 23:11:51,055][44958] Updated weights for policy 0, policy_version 75260 (0.0010) [2023-10-12 23:11:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154501120. Throughput: 0: 1647.5, 1: 1650.4. Samples: 38625836. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:11:51,444][43579] Avg episode reward: [(0, '282.570'), (1, '286.030')] [2023-10-12 23:11:54,094][44959] Updated weights for policy 1, policy_version 75620 (0.0007) [2023-10-12 23:11:54,467][44959] Updated weights for policy 1, policy_version 75630 (0.0007) [2023-10-12 23:11:54,829][44959] Updated weights for policy 1, policy_version 75640 (0.0007) [2023-10-12 23:11:55,145][44958] Updated weights for policy 0, policy_version 75270 (0.0010) [2023-10-12 23:11:55,516][44958] Updated weights for policy 0, policy_version 75280 (0.0010) [2023-10-12 23:11:55,881][44958] Updated weights for policy 0, policy_version 75290 (0.0007) [2023-10-12 23:11:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 154566656. Throughput: 0: 1640.0, 1: 1643.8. Samples: 38644816. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:11:56,443][43579] Avg episode reward: [(0, '281.910'), (1, '284.070')] [2023-10-12 23:11:59,051][44959] Updated weights for policy 1, policy_version 75650 (0.0007) [2023-10-12 23:11:59,428][44959] Updated weights for policy 1, policy_version 75660 (0.0008) [2023-10-12 23:11:59,792][44959] Updated weights for policy 1, policy_version 75670 (0.0008) [2023-10-12 23:12:00,093][44958] Updated weights for policy 0, policy_version 75300 (0.0008) [2023-10-12 23:12:00,160][44959] Updated weights for policy 1, policy_version 75680 (0.0009) [2023-10-12 23:12:00,464][44958] Updated weights for policy 0, policy_version 75310 (0.0011) [2023-10-12 23:12:00,838][44958] Updated weights for policy 0, policy_version 75320 (0.0011) [2023-10-12 23:12:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154632192. Throughput: 0: 1642.1, 1: 1655.1. Samples: 38664026. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) [2023-10-12 23:12:01,444][43579] Avg episode reward: [(0, '276.500'), (1, '280.150')] [2023-10-12 23:12:04,363][44959] Updated weights for policy 1, policy_version 75690 (0.0009) [2023-10-12 23:12:04,733][44959] Updated weights for policy 1, policy_version 75700 (0.0007) [2023-10-12 23:12:05,089][44959] Updated weights for policy 1, policy_version 75710 (0.0007) [2023-10-12 23:12:05,150][44958] Updated weights for policy 0, policy_version 75330 (0.0009) [2023-10-12 23:12:05,512][44958] Updated weights for policy 0, policy_version 75340 (0.0009) [2023-10-12 23:12:05,884][44958] Updated weights for policy 0, policy_version 75350 (0.0011) [2023-10-12 23:12:06,253][44958] Updated weights for policy 0, policy_version 75360 (0.0010) [2023-10-12 23:12:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154697728. Throughput: 0: 1646.3, 1: 1646.5. Samples: 38675242. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:06,443][43579] Avg episode reward: [(0, '273.100'), (1, '273.990')] [2023-10-12 23:12:09,337][44959] Updated weights for policy 1, policy_version 75720 (0.0009) [2023-10-12 23:12:09,708][44959] Updated weights for policy 1, policy_version 75730 (0.0009) [2023-10-12 23:12:10,072][44959] Updated weights for policy 1, policy_version 75740 (0.0009) [2023-10-12 23:12:10,357][44958] Updated weights for policy 0, policy_version 75370 (0.0008) [2023-10-12 23:12:10,733][44958] Updated weights for policy 0, policy_version 75380 (0.0007) [2023-10-12 23:12:11,109][44958] Updated weights for policy 0, policy_version 75390 (0.0008) [2023-10-12 23:12:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154763264. Throughput: 0: 1644.3, 1: 1644.1. Samples: 38694238. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:11,443][43579] Avg episode reward: [(0, '271.310'), (1, '269.470')] [2023-10-12 23:12:14,184][44959] Updated weights for policy 1, policy_version 75750 (0.0009) [2023-10-12 23:12:14,555][44959] Updated weights for policy 1, policy_version 75760 (0.0008) [2023-10-12 23:12:14,922][44959] Updated weights for policy 1, policy_version 75770 (0.0010) [2023-10-12 23:12:15,339][44958] Updated weights for policy 0, policy_version 75400 (0.0009) [2023-10-12 23:12:15,707][44958] Updated weights for policy 0, policy_version 75410 (0.0008) [2023-10-12 23:12:16,083][44958] Updated weights for policy 0, policy_version 75420 (0.0009) [2023-10-12 23:12:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154828800. Throughput: 0: 1643.2, 1: 1639.1. Samples: 38713326. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:16,444][43579] Avg episode reward: [(0, '272.190'), (1, '268.940')] [2023-10-12 23:12:19,025][44959] Updated weights for policy 1, policy_version 75780 (0.0009) [2023-10-12 23:12:19,437][44959] Updated weights for policy 1, policy_version 75790 (0.0008) [2023-10-12 23:12:19,802][44959] Updated weights for policy 1, policy_version 75800 (0.0008) [2023-10-12 23:12:20,283][44958] Updated weights for policy 0, policy_version 75430 (0.0007) [2023-10-12 23:12:20,653][44958] Updated weights for policy 0, policy_version 75440 (0.0007) [2023-10-12 23:12:21,032][44958] Updated weights for policy 0, policy_version 75450 (0.0008) [2023-10-12 23:12:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154894336. Throughput: 0: 1637.5, 1: 1634.1. Samples: 38724132. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:21,444][43579] Avg episode reward: [(0, '273.070'), (1, '272.920')] [2023-10-12 23:12:24,154][44959] Updated weights for policy 1, policy_version 75810 (0.0007) [2023-10-12 23:12:24,528][44959] Updated weights for policy 1, policy_version 75820 (0.0007) [2023-10-12 23:12:24,889][44959] Updated weights for policy 1, policy_version 75830 (0.0007) [2023-10-12 23:12:25,201][44958] Updated weights for policy 0, policy_version 75460 (0.0009) [2023-10-12 23:12:25,259][44959] Updated weights for policy 1, policy_version 75840 (0.0007) [2023-10-12 23:12:25,577][44958] Updated weights for policy 0, policy_version 75470 (0.0010) [2023-10-12 23:12:25,945][44958] Updated weights for policy 0, policy_version 75480 (0.0010) [2023-10-12 23:12:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 154959872. Throughput: 0: 1635.7, 1: 1644.4. Samples: 38743204. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:26,443][43579] Avg episode reward: [(0, '278.090'), (1, '276.040')] [2023-10-12 23:12:29,304][44959] Updated weights for policy 1, policy_version 75850 (0.0010) [2023-10-12 23:12:29,670][44959] Updated weights for policy 1, policy_version 75860 (0.0008) [2023-10-12 23:12:30,048][44959] Updated weights for policy 1, policy_version 75870 (0.0009) [2023-10-12 23:12:30,068][44958] Updated weights for policy 0, policy_version 75490 (0.0009) [2023-10-12 23:12:30,435][44958] Updated weights for policy 0, policy_version 75500 (0.0008) [2023-10-12 23:12:30,796][44958] Updated weights for policy 0, policy_version 75510 (0.0009) [2023-10-12 23:12:31,166][44958] Updated weights for policy 0, policy_version 75520 (0.0009) [2023-10-12 23:12:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155025408. Throughput: 0: 1628.8, 1: 1645.3. Samples: 38762140. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:31,443][43579] Avg episode reward: [(0, '277.870'), (1, '278.690')] [2023-10-12 23:12:34,248][44959] Updated weights for policy 1, policy_version 75880 (0.0008) [2023-10-12 23:12:34,618][44959] Updated weights for policy 1, policy_version 75890 (0.0010) [2023-10-12 23:12:34,992][44959] Updated weights for policy 1, policy_version 75900 (0.0010) [2023-10-12 23:12:35,339][44958] Updated weights for policy 0, policy_version 75530 (0.0009) [2023-10-12 23:12:35,716][44958] Updated weights for policy 0, policy_version 75540 (0.0008) [2023-10-12 23:12:36,081][44958] Updated weights for policy 0, policy_version 75550 (0.0007) [2023-10-12 23:12:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155090944. Throughput: 0: 1631.1, 1: 1643.5. Samples: 38773194. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:36,444][43579] Avg episode reward: [(0, '278.880'), (1, '282.790')] [2023-10-12 23:12:39,281][44959] Updated weights for policy 1, policy_version 75910 (0.0008) [2023-10-12 23:12:39,648][44959] Updated weights for policy 1, policy_version 75920 (0.0008) [2023-10-12 23:12:40,008][44959] Updated weights for policy 1, policy_version 75930 (0.0008) [2023-10-12 23:12:40,289][44958] Updated weights for policy 0, policy_version 75560 (0.0008) [2023-10-12 23:12:40,666][44958] Updated weights for policy 0, policy_version 75570 (0.0010) [2023-10-12 23:12:41,047][44958] Updated weights for policy 0, policy_version 75580 (0.0010) [2023-10-12 23:12:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155156480. Throughput: 0: 1632.5, 1: 1644.5. Samples: 38792284. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:41,443][43579] Avg episode reward: [(0, '278.640'), (1, '280.910')] [2023-10-12 23:12:44,096][44959] Updated weights for policy 1, policy_version 75940 (0.0008) [2023-10-12 23:12:44,460][44959] Updated weights for policy 1, policy_version 75950 (0.0007) [2023-10-12 23:12:44,836][44959] Updated weights for policy 1, policy_version 75960 (0.0007) [2023-10-12 23:12:45,347][44958] Updated weights for policy 0, policy_version 75590 (0.0009) [2023-10-12 23:12:45,704][44958] Updated weights for policy 0, policy_version 75600 (0.0010) [2023-10-12 23:12:46,075][44958] Updated weights for policy 0, policy_version 75610 (0.0010) [2023-10-12 23:12:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155222016. Throughput: 0: 1634.0, 1: 1644.8. Samples: 38811574. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:46,443][43579] Avg episode reward: [(0, '271.990'), (1, '277.870')] [2023-10-12 23:12:49,043][44959] Updated weights for policy 1, policy_version 75970 (0.0008) [2023-10-12 23:12:49,415][44959] Updated weights for policy 1, policy_version 75980 (0.0008) [2023-10-12 23:12:49,790][44959] Updated weights for policy 1, policy_version 75990 (0.0009) [2023-10-12 23:12:50,164][44959] Updated weights for policy 1, policy_version 76000 (0.0010) [2023-10-12 23:12:50,401][44958] Updated weights for policy 0, policy_version 75620 (0.0009) [2023-10-12 23:12:50,771][44958] Updated weights for policy 0, policy_version 75630 (0.0009) [2023-10-12 23:12:51,145][44958] Updated weights for policy 0, policy_version 75640 (0.0007) [2023-10-12 23:12:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155287552. Throughput: 0: 1626.7, 1: 1641.4. Samples: 38822306. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-12 23:12:51,443][43579] Avg episode reward: [(0, '267.670'), (1, '275.450')] [2023-10-12 23:12:54,363][44959] Updated weights for policy 1, policy_version 76010 (0.0007) [2023-10-12 23:12:54,727][44959] Updated weights for policy 1, policy_version 76020 (0.0008) [2023-10-12 23:12:55,093][44959] Updated weights for policy 1, policy_version 76030 (0.0007) [2023-10-12 23:12:55,178][44958] Updated weights for policy 0, policy_version 75650 (0.0008) [2023-10-12 23:12:55,559][44958] Updated weights for policy 0, policy_version 75660 (0.0009) [2023-10-12 23:12:55,928][44958] Updated weights for policy 0, policy_version 75670 (0.0008) [2023-10-12 23:12:56,302][44958] Updated weights for policy 0, policy_version 75680 (0.0009) [2023-10-12 23:12:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155353088. Throughput: 0: 1631.3, 1: 1644.2. Samples: 38841636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:12:56,444][43579] Avg episode reward: [(0, '274.150'), (1, '275.850')] [2023-10-12 23:12:59,117][44959] Updated weights for policy 1, policy_version 76040 (0.0007) [2023-10-12 23:12:59,482][44959] Updated weights for policy 1, policy_version 76050 (0.0008) [2023-10-12 23:12:59,845][44959] Updated weights for policy 1, policy_version 76060 (0.0010) [2023-10-12 23:13:00,456][44958] Updated weights for policy 0, policy_version 75690 (0.0009) [2023-10-12 23:13:00,828][44958] Updated weights for policy 0, policy_version 75700 (0.0007) [2023-10-12 23:13:01,204][44958] Updated weights for policy 0, policy_version 75710 (0.0007) [2023-10-12 23:13:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155418624. Throughput: 0: 1632.7, 1: 1645.4. Samples: 38860840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:01,443][43579] Avg episode reward: [(0, '277.570'), (1, '277.700')] [2023-10-12 23:13:04,236][44959] Updated weights for policy 1, policy_version 76070 (0.0008) [2023-10-12 23:13:04,622][44959] Updated weights for policy 1, policy_version 76080 (0.0010) [2023-10-12 23:13:04,990][44959] Updated weights for policy 1, policy_version 76090 (0.0008) [2023-10-12 23:13:05,153][44958] Updated weights for policy 0, policy_version 75720 (0.0009) [2023-10-12 23:13:05,522][44958] Updated weights for policy 0, policy_version 75730 (0.0008) [2023-10-12 23:13:05,895][44958] Updated weights for policy 0, policy_version 75740 (0.0011) [2023-10-12 23:13:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155484160. Throughput: 0: 1637.2, 1: 1648.5. Samples: 38871992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:06,444][43579] Avg episode reward: [(0, '274.230'), (1, '272.590')] [2023-10-12 23:13:09,160][44959] Updated weights for policy 1, policy_version 76100 (0.0008) [2023-10-12 23:13:09,534][44959] Updated weights for policy 1, policy_version 76110 (0.0009) [2023-10-12 23:13:09,904][44959] Updated weights for policy 1, policy_version 76120 (0.0008) [2023-10-12 23:13:10,225][44958] Updated weights for policy 0, policy_version 75750 (0.0008) [2023-10-12 23:13:10,605][44958] Updated weights for policy 0, policy_version 75760 (0.0008) [2023-10-12 23:13:10,971][44958] Updated weights for policy 0, policy_version 75770 (0.0008) [2023-10-12 23:13:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155549696. Throughput: 0: 1638.4, 1: 1646.3. Samples: 38891016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:11,444][43579] Avg episode reward: [(0, '275.030'), (1, '283.810')] [2023-10-12 23:13:13,928][44959] Updated weights for policy 1, policy_version 76130 (0.0007) [2023-10-12 23:13:14,292][44959] Updated weights for policy 1, policy_version 76140 (0.0008) [2023-10-12 23:13:14,664][44959] Updated weights for policy 1, policy_version 76150 (0.0008) [2023-10-12 23:13:15,031][44959] Updated weights for policy 1, policy_version 76160 (0.0008) [2023-10-12 23:13:15,110][44958] Updated weights for policy 0, policy_version 75780 (0.0008) [2023-10-12 23:13:15,475][44958] Updated weights for policy 0, policy_version 75790 (0.0008) [2023-10-12 23:13:15,844][44958] Updated weights for policy 0, policy_version 75800 (0.0010) [2023-10-12 23:13:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155615232. Throughput: 0: 1640.7, 1: 1644.5. Samples: 38909972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:16,443][43579] Avg episode reward: [(0, '278.510'), (1, '285.700')] [2023-10-12 23:13:19,209][44959] Updated weights for policy 1, policy_version 76170 (0.0009) [2023-10-12 23:13:19,570][44959] Updated weights for policy 1, policy_version 76180 (0.0010) [2023-10-12 23:13:19,942][44958] Updated weights for policy 0, policy_version 75810 (0.0007) [2023-10-12 23:13:19,950][44959] Updated weights for policy 1, policy_version 76190 (0.0009) [2023-10-12 23:13:20,315][44958] Updated weights for policy 0, policy_version 75820 (0.0009) [2023-10-12 23:13:20,686][44958] Updated weights for policy 0, policy_version 75830 (0.0008) [2023-10-12 23:13:21,060][44958] Updated weights for policy 0, policy_version 75840 (0.0008) [2023-10-12 23:13:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155680768. Throughput: 0: 1642.0, 1: 1644.5. Samples: 38921086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:21,444][43579] Avg episode reward: [(0, '277.130'), (1, '288.320')] [2023-10-12 23:13:24,008][44959] Updated weights for policy 1, policy_version 76200 (0.0008) [2023-10-12 23:13:24,374][44959] Updated weights for policy 1, policy_version 76210 (0.0007) [2023-10-12 23:13:24,750][44959] Updated weights for policy 1, policy_version 76220 (0.0008) [2023-10-12 23:13:25,576][44958] Updated weights for policy 0, policy_version 75850 (0.0010) [2023-10-12 23:13:25,956][44958] Updated weights for policy 0, policy_version 75860 (0.0010) [2023-10-12 23:13:26,335][44958] Updated weights for policy 0, policy_version 75870 (0.0009) [2023-10-12 23:13:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155746304. Throughput: 0: 1643.4, 1: 1640.7. Samples: 38940068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:26,443][43579] Avg episode reward: [(0, '272.140'), (1, '287.890')] [2023-10-12 23:13:28,888][44959] Updated weights for policy 1, policy_version 76230 (0.0008) [2023-10-12 23:13:29,264][44959] Updated weights for policy 1, policy_version 76240 (0.0008) [2023-10-12 23:13:29,635][44959] Updated weights for policy 1, policy_version 76250 (0.0009) [2023-10-12 23:13:30,507][44958] Updated weights for policy 0, policy_version 75880 (0.0009) [2023-10-12 23:13:30,870][44958] Updated weights for policy 0, policy_version 75890 (0.0010) [2023-10-12 23:13:31,246][44958] Updated weights for policy 0, policy_version 75900 (0.0010) [2023-10-12 23:13:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155811840. Throughput: 0: 1639.7, 1: 1643.0. Samples: 38959294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:31,444][43579] Avg episode reward: [(0, '269.020'), (1, '285.310')] [2023-10-12 23:13:33,967][44959] Updated weights for policy 1, policy_version 76260 (0.0009) [2023-10-12 23:13:34,354][44959] Updated weights for policy 1, policy_version 76270 (0.0009) [2023-10-12 23:13:34,726][44959] Updated weights for policy 1, policy_version 76280 (0.0010) [2023-10-12 23:13:35,418][44958] Updated weights for policy 0, policy_version 75910 (0.0010) [2023-10-12 23:13:35,786][44958] Updated weights for policy 0, policy_version 75920 (0.0009) [2023-10-12 23:13:36,157][44958] Updated weights for policy 0, policy_version 75930 (0.0009) [2023-10-12 23:13:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 155877376. Throughput: 0: 1640.5, 1: 1640.5. Samples: 38969954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:36,444][43579] Avg episode reward: [(0, '273.270'), (1, '290.340')] [2023-10-12 23:13:38,807][44959] Updated weights for policy 1, policy_version 76290 (0.0007) [2023-10-12 23:13:39,172][44959] Updated weights for policy 1, policy_version 76300 (0.0007) [2023-10-12 23:13:39,531][44959] Updated weights for policy 1, policy_version 76310 (0.0007) [2023-10-12 23:13:39,911][44959] Updated weights for policy 1, policy_version 76320 (0.0007) [2023-10-12 23:13:40,445][44958] Updated weights for policy 0, policy_version 75940 (0.0008) [2023-10-12 23:13:40,816][44958] Updated weights for policy 0, policy_version 75950 (0.0007) [2023-10-12 23:13:41,185][44958] Updated weights for policy 0, policy_version 75960 (0.0008) [2023-10-12 23:13:41,442][43579] Fps is (10 sec: 9830.6, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 155910144. Throughput: 0: 1642.5, 1: 1637.3. Samples: 38989226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:41,443][43579] Avg episode reward: [(0, '266.810'), (1, '289.960')] [2023-10-12 23:13:44,152][44959] Updated weights for policy 1, policy_version 76330 (0.0007) [2023-10-12 23:13:44,518][44959] Updated weights for policy 1, policy_version 76340 (0.0010) [2023-10-12 23:13:44,887][44959] Updated weights for policy 1, policy_version 76350 (0.0008) [2023-10-12 23:13:45,143][44958] Updated weights for policy 0, policy_version 75970 (0.0008) [2023-10-12 23:13:45,519][44958] Updated weights for policy 0, policy_version 75980 (0.0008) [2023-10-12 23:13:45,892][44958] Updated weights for policy 0, policy_version 75990 (0.0009) [2023-10-12 23:13:46,258][44958] Updated weights for policy 0, policy_version 76000 (0.0009) [2023-10-12 23:13:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156008448. Throughput: 0: 1641.2, 1: 1636.9. Samples: 39008358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:46,444][43579] Avg episode reward: [(0, '262.270'), (1, '285.290')] [2023-10-12 23:13:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000076000_77824000.pth... [2023-10-12 23:13:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000076352_78184448.pth... [2023-10-12 23:13:46,491][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000074816_76611584.pth [2023-10-12 23:13:46,496][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000074464_76251136.pth [2023-10-12 23:13:49,018][44959] Updated weights for policy 1, policy_version 76360 (0.0007) [2023-10-12 23:13:49,387][44959] Updated weights for policy 1, policy_version 76370 (0.0011) [2023-10-12 23:13:49,763][44959] Updated weights for policy 1, policy_version 76380 (0.0008) [2023-10-12 23:13:50,441][44958] Updated weights for policy 0, policy_version 76010 (0.0008) [2023-10-12 23:13:50,830][44958] Updated weights for policy 0, policy_version 76020 (0.0008) [2023-10-12 23:13:51,204][44958] Updated weights for policy 0, policy_version 76030 (0.0008) [2023-10-12 23:13:51,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156073984. Throughput: 0: 1639.2, 1: 1634.7. Samples: 39019316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:51,443][43579] Avg episode reward: [(0, '270.470'), (1, '278.300')] [2023-10-12 23:13:53,812][44959] Updated weights for policy 1, policy_version 76390 (0.0007) [2023-10-12 23:13:54,170][44959] Updated weights for policy 1, policy_version 76400 (0.0009) [2023-10-12 23:13:54,544][44959] Updated weights for policy 1, policy_version 76410 (0.0007) [2023-10-12 23:13:55,463][44958] Updated weights for policy 0, policy_version 76040 (0.0008) [2023-10-12 23:13:55,836][44958] Updated weights for policy 0, policy_version 76050 (0.0008) [2023-10-12 23:13:56,215][44958] Updated weights for policy 0, policy_version 76060 (0.0008) [2023-10-12 23:13:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156139520. Throughput: 0: 1636.7, 1: 1641.0. Samples: 39038512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:13:56,443][43579] Avg episode reward: [(0, '271.870'), (1, '278.630')] [2023-10-12 23:13:58,852][44959] Updated weights for policy 1, policy_version 76420 (0.0008) [2023-10-12 23:13:59,256][44959] Updated weights for policy 1, policy_version 76430 (0.0008) [2023-10-12 23:13:59,631][44959] Updated weights for policy 1, policy_version 76440 (0.0008) [2023-10-12 23:14:00,220][44958] Updated weights for policy 0, policy_version 76070 (0.0010) [2023-10-12 23:14:00,588][44958] Updated weights for policy 0, policy_version 76080 (0.0008) [2023-10-12 23:14:00,953][44958] Updated weights for policy 0, policy_version 76090 (0.0009) [2023-10-12 23:14:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156205056. Throughput: 0: 1642.0, 1: 1642.9. Samples: 39057792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:01,443][43579] Avg episode reward: [(0, '270.290'), (1, '275.050')] [2023-10-12 23:14:03,699][44959] Updated weights for policy 1, policy_version 76450 (0.0008) [2023-10-12 23:14:04,064][44959] Updated weights for policy 1, policy_version 76460 (0.0008) [2023-10-12 23:14:04,429][44959] Updated weights for policy 1, policy_version 76470 (0.0007) [2023-10-12 23:14:04,796][44959] Updated weights for policy 1, policy_version 76480 (0.0008) [2023-10-12 23:14:05,129][44958] Updated weights for policy 0, policy_version 76100 (0.0008) [2023-10-12 23:14:05,500][44958] Updated weights for policy 0, policy_version 76110 (0.0008) [2023-10-12 23:14:05,876][44958] Updated weights for policy 0, policy_version 76120 (0.0007) [2023-10-12 23:14:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156270592. Throughput: 0: 1641.9, 1: 1638.6. Samples: 39068706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:06,444][43579] Avg episode reward: [(0, '268.880'), (1, '275.370')] [2023-10-12 23:14:08,946][44959] Updated weights for policy 1, policy_version 76490 (0.0008) [2023-10-12 23:14:09,309][44959] Updated weights for policy 1, policy_version 76500 (0.0009) [2023-10-12 23:14:09,677][44959] Updated weights for policy 1, policy_version 76510 (0.0008) [2023-10-12 23:14:10,124][44958] Updated weights for policy 0, policy_version 76130 (0.0008) [2023-10-12 23:14:10,525][44958] Updated weights for policy 0, policy_version 76140 (0.0009) [2023-10-12 23:14:10,896][44958] Updated weights for policy 0, policy_version 76150 (0.0009) [2023-10-12 23:14:11,264][44958] Updated weights for policy 0, policy_version 76160 (0.0008) [2023-10-12 23:14:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156336128. Throughput: 0: 1643.0, 1: 1647.3. Samples: 39088130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:11,444][43579] Avg episode reward: [(0, '280.880'), (1, '277.920')] [2023-10-12 23:14:13,843][44959] Updated weights for policy 1, policy_version 76520 (0.0009) [2023-10-12 23:14:14,208][44959] Updated weights for policy 1, policy_version 76530 (0.0009) [2023-10-12 23:14:14,585][44959] Updated weights for policy 1, policy_version 76540 (0.0009) [2023-10-12 23:14:15,400][44958] Updated weights for policy 0, policy_version 76170 (0.0008) [2023-10-12 23:14:15,765][44958] Updated weights for policy 0, policy_version 76180 (0.0010) [2023-10-12 23:14:16,145][44958] Updated weights for policy 0, policy_version 76190 (0.0010) [2023-10-12 23:14:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156401664. Throughput: 0: 1642.2, 1: 1643.3. Samples: 39107144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:16,443][43579] Avg episode reward: [(0, '274.930'), (1, '284.090')] [2023-10-12 23:14:18,723][44959] Updated weights for policy 1, policy_version 76550 (0.0010) [2023-10-12 23:14:19,092][44959] Updated weights for policy 1, policy_version 76560 (0.0011) [2023-10-12 23:14:19,458][44959] Updated weights for policy 1, policy_version 76570 (0.0011) [2023-10-12 23:14:20,163][44958] Updated weights for policy 0, policy_version 76200 (0.0008) [2023-10-12 23:14:20,535][44958] Updated weights for policy 0, policy_version 76210 (0.0008) [2023-10-12 23:14:20,911][44958] Updated weights for policy 0, policy_version 76220 (0.0010) [2023-10-12 23:14:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156467200. Throughput: 0: 1648.3, 1: 1640.3. Samples: 39117942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:21,444][43579] Avg episode reward: [(0, '275.830'), (1, '286.460')] [2023-10-12 23:14:23,740][44959] Updated weights for policy 1, policy_version 76580 (0.0008) [2023-10-12 23:14:24,106][44959] Updated weights for policy 1, policy_version 76590 (0.0007) [2023-10-12 23:14:24,467][44959] Updated weights for policy 1, policy_version 76600 (0.0007) [2023-10-12 23:14:25,166][44958] Updated weights for policy 0, policy_version 76230 (0.0008) [2023-10-12 23:14:25,542][44958] Updated weights for policy 0, policy_version 76240 (0.0008) [2023-10-12 23:14:25,904][44958] Updated weights for policy 0, policy_version 76250 (0.0009) [2023-10-12 23:14:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156532736. Throughput: 0: 1641.4, 1: 1646.3. Samples: 39137174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:26,444][43579] Avg episode reward: [(0, '277.070'), (1, '285.000')] [2023-10-12 23:14:28,514][44959] Updated weights for policy 1, policy_version 76610 (0.0007) [2023-10-12 23:14:28,885][44959] Updated weights for policy 1, policy_version 76620 (0.0007) [2023-10-12 23:14:29,243][44959] Updated weights for policy 1, policy_version 76630 (0.0007) [2023-10-12 23:14:29,610][44959] Updated weights for policy 1, policy_version 76640 (0.0008) [2023-10-12 23:14:29,969][44958] Updated weights for policy 0, policy_version 76260 (0.0009) [2023-10-12 23:14:30,337][44958] Updated weights for policy 0, policy_version 76270 (0.0007) [2023-10-12 23:14:30,702][44958] Updated weights for policy 0, policy_version 76280 (0.0007) [2023-10-12 23:14:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 156598272. Throughput: 0: 1640.9, 1: 1652.8. Samples: 39156572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:31,443][43579] Avg episode reward: [(0, '278.210'), (1, '286.170')] [2023-10-12 23:14:33,718][44959] Updated weights for policy 1, policy_version 76650 (0.0008) [2023-10-12 23:14:34,088][44959] Updated weights for policy 1, policy_version 76660 (0.0009) [2023-10-12 23:14:34,451][44959] Updated weights for policy 1, policy_version 76670 (0.0008) [2023-10-12 23:14:34,683][44958] Updated weights for policy 0, policy_version 76290 (0.0010) [2023-10-12 23:14:35,050][44958] Updated weights for policy 0, policy_version 76300 (0.0011) [2023-10-12 23:14:35,424][44958] Updated weights for policy 0, policy_version 76310 (0.0008) [2023-10-12 23:14:35,797][44958] Updated weights for policy 0, policy_version 76320 (0.0010) [2023-10-12 23:14:36,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156663808. Throughput: 0: 1646.1, 1: 1644.2. Samples: 39167382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:36,443][43579] Avg episode reward: [(0, '276.910'), (1, '284.020')] [2023-10-12 23:14:38,631][44959] Updated weights for policy 1, policy_version 76680 (0.0009) [2023-10-12 23:14:39,009][44959] Updated weights for policy 1, policy_version 76690 (0.0007) [2023-10-12 23:14:39,385][44959] Updated weights for policy 1, policy_version 76700 (0.0009) [2023-10-12 23:14:40,065][44958] Updated weights for policy 0, policy_version 76330 (0.0008) [2023-10-12 23:14:40,446][44958] Updated weights for policy 0, policy_version 76340 (0.0008) [2023-10-12 23:14:40,818][44958] Updated weights for policy 0, policy_version 76350 (0.0007) [2023-10-12 23:14:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 156729344. Throughput: 0: 1643.6, 1: 1648.1. Samples: 39186638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:41,443][43579] Avg episode reward: [(0, '275.080'), (1, '284.090')] [2023-10-12 23:14:43,652][44959] Updated weights for policy 1, policy_version 76710 (0.0007) [2023-10-12 23:14:44,041][44959] Updated weights for policy 1, policy_version 76720 (0.0008) [2023-10-12 23:14:44,407][44959] Updated weights for policy 1, policy_version 76730 (0.0008) [2023-10-12 23:14:45,033][44958] Updated weights for policy 0, policy_version 76360 (0.0008) [2023-10-12 23:14:45,401][44958] Updated weights for policy 0, policy_version 76370 (0.0008) [2023-10-12 23:14:45,768][44958] Updated weights for policy 0, policy_version 76380 (0.0008) [2023-10-12 23:14:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156794880. Throughput: 0: 1649.8, 1: 1651.4. Samples: 39206346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:46,444][43579] Avg episode reward: [(0, '271.420'), (1, '282.230')] [2023-10-12 23:14:48,474][44959] Updated weights for policy 1, policy_version 76740 (0.0008) [2023-10-12 23:14:48,843][44959] Updated weights for policy 1, policy_version 76750 (0.0009) [2023-10-12 23:14:49,212][44959] Updated weights for policy 1, policy_version 76760 (0.0009) [2023-10-12 23:14:50,090][44958] Updated weights for policy 0, policy_version 76390 (0.0010) [2023-10-12 23:14:50,462][44958] Updated weights for policy 0, policy_version 76400 (0.0008) [2023-10-12 23:14:50,826][44958] Updated weights for policy 0, policy_version 76410 (0.0011) [2023-10-12 23:14:51,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156860416. Throughput: 0: 1647.7, 1: 1645.5. Samples: 39216902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:51,444][43579] Avg episode reward: [(0, '271.680'), (1, '283.570')] [2023-10-12 23:14:53,457][44959] Updated weights for policy 1, policy_version 76770 (0.0009) [2023-10-12 23:14:53,825][44959] Updated weights for policy 1, policy_version 76780 (0.0008) [2023-10-12 23:14:54,196][44959] Updated weights for policy 1, policy_version 76790 (0.0008) [2023-10-12 23:14:54,575][44959] Updated weights for policy 1, policy_version 76800 (0.0008) [2023-10-12 23:14:55,253][44958] Updated weights for policy 0, policy_version 76420 (0.0007) [2023-10-12 23:14:55,645][44958] Updated weights for policy 0, policy_version 76430 (0.0010) [2023-10-12 23:14:56,020][44958] Updated weights for policy 0, policy_version 76440 (0.0009) [2023-10-12 23:14:56,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156925952. Throughput: 0: 1642.7, 1: 1648.0. Samples: 39236212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:14:56,443][43579] Avg episode reward: [(0, '271.610'), (1, '287.560')] [2023-10-12 23:14:58,712][44959] Updated weights for policy 1, policy_version 76810 (0.0011) [2023-10-12 23:14:59,093][44959] Updated weights for policy 1, policy_version 76820 (0.0010) [2023-10-12 23:14:59,449][44959] Updated weights for policy 1, policy_version 76830 (0.0010) [2023-10-12 23:14:59,973][44958] Updated weights for policy 0, policy_version 76450 (0.0008) [2023-10-12 23:15:00,352][44958] Updated weights for policy 0, policy_version 76460 (0.0011) [2023-10-12 23:15:00,729][44958] Updated weights for policy 0, policy_version 76470 (0.0007) [2023-10-12 23:15:01,098][44958] Updated weights for policy 0, policy_version 76480 (0.0007) [2023-10-12 23:15:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 156991488. Throughput: 0: 1642.4, 1: 1652.6. Samples: 39255418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:15:01,443][43579] Avg episode reward: [(0, '277.200'), (1, '291.190')] [2023-10-12 23:15:03,622][44959] Updated weights for policy 1, policy_version 76840 (0.0008) [2023-10-12 23:15:03,987][44959] Updated weights for policy 1, policy_version 76850 (0.0008) [2023-10-12 23:15:04,360][44959] Updated weights for policy 1, policy_version 76860 (0.0008) [2023-10-12 23:15:04,989][44958] Updated weights for policy 0, policy_version 76490 (0.0008) [2023-10-12 23:15:05,368][44958] Updated weights for policy 0, policy_version 76500 (0.0008) [2023-10-12 23:15:05,737][44958] Updated weights for policy 0, policy_version 76510 (0.0009) [2023-10-12 23:15:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157057024. Throughput: 0: 1646.8, 1: 1645.2. Samples: 39266084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:15:06,443][43579] Avg episode reward: [(0, '275.310'), (1, '290.200')] [2023-10-12 23:15:08,609][44959] Updated weights for policy 1, policy_version 76870 (0.0009) [2023-10-12 23:15:08,979][44959] Updated weights for policy 1, policy_version 76880 (0.0008) [2023-10-12 23:15:09,346][44959] Updated weights for policy 1, policy_version 76890 (0.0009) [2023-10-12 23:15:09,927][44958] Updated weights for policy 0, policy_version 76520 (0.0007) [2023-10-12 23:15:10,299][44958] Updated weights for policy 0, policy_version 76530 (0.0007) [2023-10-12 23:15:10,678][44958] Updated weights for policy 0, policy_version 76540 (0.0007) [2023-10-12 23:15:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157122560. Throughput: 0: 1644.4, 1: 1648.4. Samples: 39285350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:15:11,443][43579] Avg episode reward: [(0, '275.300'), (1, '288.580')] [2023-10-12 23:15:13,429][44959] Updated weights for policy 1, policy_version 76900 (0.0008) [2023-10-12 23:15:13,791][44959] Updated weights for policy 1, policy_version 76910 (0.0009) [2023-10-12 23:15:14,162][44959] Updated weights for policy 1, policy_version 76920 (0.0009) [2023-10-12 23:15:14,677][44958] Updated weights for policy 0, policy_version 76550 (0.0009) [2023-10-12 23:15:15,040][44958] Updated weights for policy 0, policy_version 76560 (0.0007) [2023-10-12 23:15:15,417][44958] Updated weights for policy 0, policy_version 76570 (0.0008) [2023-10-12 23:15:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157188096. Throughput: 0: 1651.8, 1: 1640.7. Samples: 39304734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:15:16,443][43579] Avg episode reward: [(0, '284.790'), (1, '289.910')] [2023-10-12 23:15:18,441][44959] Updated weights for policy 1, policy_version 76930 (0.0009) [2023-10-12 23:15:18,813][44959] Updated weights for policy 1, policy_version 76940 (0.0009) [2023-10-12 23:15:19,188][44959] Updated weights for policy 1, policy_version 76950 (0.0008) [2023-10-12 23:15:19,565][44959] Updated weights for policy 1, policy_version 76960 (0.0010) [2023-10-12 23:15:19,699][44958] Updated weights for policy 0, policy_version 76580 (0.0010) [2023-10-12 23:15:20,075][44958] Updated weights for policy 0, policy_version 76590 (0.0008) [2023-10-12 23:15:20,462][44958] Updated weights for policy 0, policy_version 76600 (0.0008) [2023-10-12 23:15:21,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157253632. Throughput: 0: 1649.9, 1: 1639.6. Samples: 39315408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:15:21,444][43579] Avg episode reward: [(0, '284.350'), (1, '291.460')] [2023-10-12 23:15:23,763][44959] Updated weights for policy 1, policy_version 76970 (0.0008) [2023-10-12 23:15:24,145][44959] Updated weights for policy 1, policy_version 76980 (0.0009) [2023-10-12 23:15:24,517][44959] Updated weights for policy 1, policy_version 76990 (0.0009) [2023-10-12 23:15:24,734][44958] Updated weights for policy 0, policy_version 76610 (0.0008) [2023-10-12 23:15:25,102][44958] Updated weights for policy 0, policy_version 76620 (0.0008) [2023-10-12 23:15:25,477][44958] Updated weights for policy 0, policy_version 76630 (0.0008) [2023-10-12 23:15:25,851][44958] Updated weights for policy 0, policy_version 76640 (0.0008) [2023-10-12 23:15:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 157319168. Throughput: 0: 1648.8, 1: 1634.3. Samples: 39334378. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:15:26,443][43579] Avg episode reward: [(0, '274.350'), (1, '291.780')] [2023-10-12 23:15:28,785][44959] Updated weights for policy 1, policy_version 77000 (0.0009) [2023-10-12 23:15:29,174][44959] Updated weights for policy 1, policy_version 77010 (0.0008) [2023-10-12 23:15:29,538][44959] Updated weights for policy 1, policy_version 77020 (0.0009) [2023-10-12 23:15:29,676][44958] Updated weights for policy 0, policy_version 76650 (0.0008) [2023-10-12 23:15:30,046][44958] Updated weights for policy 0, policy_version 76660 (0.0011) [2023-10-12 23:15:30,417][44958] Updated weights for policy 0, policy_version 76670 (0.0009) [2023-10-12 23:15:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157384704. Throughput: 0: 1649.5, 1: 1628.7. Samples: 39353864. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:15:31,443][43579] Avg episode reward: [(0, '271.740'), (1, '289.370')] [2023-10-12 23:15:33,558][44959] Updated weights for policy 1, policy_version 77030 (0.0009) [2023-10-12 23:15:33,931][44959] Updated weights for policy 1, policy_version 77040 (0.0010) [2023-10-12 23:15:34,301][44959] Updated weights for policy 1, policy_version 77050 (0.0009) [2023-10-12 23:15:34,766][44958] Updated weights for policy 0, policy_version 76680 (0.0007) [2023-10-12 23:15:35,138][44958] Updated weights for policy 0, policy_version 76690 (0.0007) [2023-10-12 23:15:35,513][44958] Updated weights for policy 0, policy_version 76700 (0.0008) [2023-10-12 23:15:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157450240. Throughput: 0: 1651.5, 1: 1627.7. Samples: 39364464. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:15:36,443][43579] Avg episode reward: [(0, '273.130'), (1, '287.960')] [2023-10-12 23:15:38,685][44959] Updated weights for policy 1, policy_version 77060 (0.0008) [2023-10-12 23:15:39,045][44959] Updated weights for policy 1, policy_version 77070 (0.0007) [2023-10-12 23:15:39,416][44959] Updated weights for policy 1, policy_version 77080 (0.0007) [2023-10-12 23:15:39,688][44958] Updated weights for policy 0, policy_version 76710 (0.0009) [2023-10-12 23:15:40,073][44958] Updated weights for policy 0, policy_version 76720 (0.0007) [2023-10-12 23:15:40,449][44958] Updated weights for policy 0, policy_version 76730 (0.0009) [2023-10-12 23:15:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157515776. Throughput: 0: 1641.1, 1: 1630.3. Samples: 39383426. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:15:41,443][43579] Avg episode reward: [(0, '276.040'), (1, '291.520')] [2023-10-12 23:15:43,480][44959] Updated weights for policy 1, policy_version 77090 (0.0008) [2023-10-12 23:15:43,841][44959] Updated weights for policy 1, policy_version 77100 (0.0010) [2023-10-12 23:15:44,209][44959] Updated weights for policy 1, policy_version 77110 (0.0008) [2023-10-12 23:15:44,564][44958] Updated weights for policy 0, policy_version 76740 (0.0009) [2023-10-12 23:15:44,576][44959] Updated weights for policy 1, policy_version 77120 (0.0010) [2023-10-12 23:15:44,938][44958] Updated weights for policy 0, policy_version 76750 (0.0009) [2023-10-12 23:15:45,321][44958] Updated weights for policy 0, policy_version 76760 (0.0008) [2023-10-12 23:15:46,443][43579] Fps is (10 sec: 13106.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157581312. Throughput: 0: 1650.3, 1: 1632.0. Samples: 39403122. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:15:46,444][43579] Avg episode reward: [(0, '275.990'), (1, '291.560')] [2023-10-12 23:15:46,456][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000076768_78610432.pth... [2023-10-12 23:15:46,457][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000077120_78970880.pth... [2023-10-12 23:15:46,490][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000075584_77398016.pth [2023-10-12 23:15:46,494][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000077120_78970880.pth [2023-10-12 23:15:46,494][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000075232_77037568.pth [2023-10-12 23:15:46,500][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000076768_78610432.pth [2023-10-12 23:15:48,848][44959] Updated weights for policy 1, policy_version 77130 (0.0010) [2023-10-12 23:15:49,211][44959] Updated weights for policy 1, policy_version 77140 (0.0009) [2023-10-12 23:15:49,577][44959] Updated weights for policy 1, policy_version 77150 (0.0007) [2023-10-12 23:15:49,731][44958] Updated weights for policy 0, policy_version 76770 (0.0010) [2023-10-12 23:15:50,105][44958] Updated weights for policy 0, policy_version 76780 (0.0009) [2023-10-12 23:15:50,471][44958] Updated weights for policy 0, policy_version 76790 (0.0008) [2023-10-12 23:15:50,851][44958] Updated weights for policy 0, policy_version 76800 (0.0007) [2023-10-12 23:15:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 157646848. Throughput: 0: 1644.2, 1: 1634.2. Samples: 39413612. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:15:51,443][43579] Avg episode reward: [(0, '276.170'), (1, '287.230')] [2023-10-12 23:15:53,787][44959] Updated weights for policy 1, policy_version 77160 (0.0008) [2023-10-12 23:15:54,167][44959] Updated weights for policy 1, policy_version 77170 (0.0009) [2023-10-12 23:15:54,526][44959] Updated weights for policy 1, policy_version 77180 (0.0007) [2023-10-12 23:15:54,792][44958] Updated weights for policy 0, policy_version 76810 (0.0010) [2023-10-12 23:15:55,167][44958] Updated weights for policy 0, policy_version 76820 (0.0011) [2023-10-12 23:15:55,540][44958] Updated weights for policy 0, policy_version 76830 (0.0011) [2023-10-12 23:15:56,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157712384. Throughput: 0: 1638.0, 1: 1635.5. Samples: 39432662. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:15:56,444][43579] Avg episode reward: [(0, '281.850'), (1, '287.520')] [2023-10-12 23:15:58,518][44959] Updated weights for policy 1, policy_version 77190 (0.0008) [2023-10-12 23:15:58,891][44959] Updated weights for policy 1, policy_version 77200 (0.0010) [2023-10-12 23:15:59,257][44959] Updated weights for policy 1, policy_version 77210 (0.0011) [2023-10-12 23:15:59,826][44958] Updated weights for policy 0, policy_version 76840 (0.0009) [2023-10-12 23:16:00,195][44958] Updated weights for policy 0, policy_version 76850 (0.0007) [2023-10-12 23:16:00,566][44958] Updated weights for policy 0, policy_version 76860 (0.0008) [2023-10-12 23:16:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 157777920. Throughput: 0: 1640.3, 1: 1643.8. Samples: 39452520. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:16:01,444][43579] Avg episode reward: [(0, '281.460'), (1, '289.160')] [2023-10-12 23:16:03,419][44959] Updated weights for policy 1, policy_version 77220 (0.0008) [2023-10-12 23:16:03,790][44959] Updated weights for policy 1, policy_version 77230 (0.0007) [2023-10-12 23:16:04,161][44959] Updated weights for policy 1, policy_version 77240 (0.0007) [2023-10-12 23:16:04,730][44958] Updated weights for policy 0, policy_version 76870 (0.0007) [2023-10-12 23:16:05,110][44958] Updated weights for policy 0, policy_version 76880 (0.0008) [2023-10-12 23:16:05,479][44958] Updated weights for policy 0, policy_version 76890 (0.0009) [2023-10-12 23:16:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157843456. Throughput: 0: 1638.3, 1: 1645.4. Samples: 39463174. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:16:06,443][43579] Avg episode reward: [(0, '284.300'), (1, '291.120')] [2023-10-12 23:16:08,435][44959] Updated weights for policy 1, policy_version 77250 (0.0008) [2023-10-12 23:16:08,802][44959] Updated weights for policy 1, policy_version 77260 (0.0009) [2023-10-12 23:16:09,175][44959] Updated weights for policy 1, policy_version 77270 (0.0009) [2023-10-12 23:16:09,526][44958] Updated weights for policy 0, policy_version 76900 (0.0009) [2023-10-12 23:16:09,536][44959] Updated weights for policy 1, policy_version 77280 (0.0009) [2023-10-12 23:16:09,909][44958] Updated weights for policy 0, policy_version 76910 (0.0007) [2023-10-12 23:16:10,283][44958] Updated weights for policy 0, policy_version 76920 (0.0007) [2023-10-12 23:16:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157908992. Throughput: 0: 1631.5, 1: 1655.7. Samples: 39482302. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-12 23:16:11,444][43579] Avg episode reward: [(0, '283.420'), (1, '288.760')] [2023-10-12 23:16:13,760][44959] Updated weights for policy 1, policy_version 77290 (0.0007) [2023-10-12 23:16:14,136][44959] Updated weights for policy 1, policy_version 77300 (0.0008) [2023-10-12 23:16:14,354][44958] Updated weights for policy 0, policy_version 76930 (0.0007) [2023-10-12 23:16:14,494][44959] Updated weights for policy 1, policy_version 77310 (0.0009) [2023-10-12 23:16:14,724][44958] Updated weights for policy 0, policy_version 76940 (0.0009) [2023-10-12 23:16:15,096][44958] Updated weights for policy 0, policy_version 76950 (0.0010) [2023-10-12 23:16:15,480][44958] Updated weights for policy 0, policy_version 76960 (0.0011) [2023-10-12 23:16:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 157974528. Throughput: 0: 1632.8, 1: 1660.7. Samples: 39502070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:16:16,443][43579] Avg episode reward: [(0, '280.670'), (1, '285.520')] [2023-10-12 23:16:18,651][44959] Updated weights for policy 1, policy_version 77320 (0.0008) [2023-10-12 23:16:19,010][44959] Updated weights for policy 1, policy_version 77330 (0.0008) [2023-10-12 23:16:19,385][44959] Updated weights for policy 1, policy_version 77340 (0.0009) [2023-10-12 23:16:19,750][44958] Updated weights for policy 0, policy_version 76970 (0.0008) [2023-10-12 23:16:20,131][44958] Updated weights for policy 0, policy_version 76980 (0.0010) [2023-10-12 23:16:20,500][44958] Updated weights for policy 0, policy_version 76990 (0.0009) [2023-10-12 23:16:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158040064. Throughput: 0: 1630.0, 1: 1657.3. Samples: 39512394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:16:21,444][43579] Avg episode reward: [(0, '279.660'), (1, '285.400')] [2023-10-12 23:16:23,467][44959] Updated weights for policy 1, policy_version 77350 (0.0009) [2023-10-12 23:16:23,839][44959] Updated weights for policy 1, policy_version 77360 (0.0007) [2023-10-12 23:16:24,197][44959] Updated weights for policy 1, policy_version 77370 (0.0009) [2023-10-12 23:16:24,833][44958] Updated weights for policy 0, policy_version 77000 (0.0008) [2023-10-12 23:16:25,216][44958] Updated weights for policy 0, policy_version 77010 (0.0008) [2023-10-12 23:16:25,593][44958] Updated weights for policy 0, policy_version 77020 (0.0007) [2023-10-12 23:16:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158105600. Throughput: 0: 1631.0, 1: 1656.8. Samples: 39531378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:16:26,443][43579] Avg episode reward: [(0, '283.330'), (1, '280.670')] [2023-10-12 23:16:28,273][44959] Updated weights for policy 1, policy_version 77380 (0.0011) [2023-10-12 23:16:28,643][44959] Updated weights for policy 1, policy_version 77390 (0.0010) [2023-10-12 23:16:29,021][44959] Updated weights for policy 1, policy_version 77400 (0.0008) [2023-10-12 23:16:29,564][44958] Updated weights for policy 0, policy_version 77030 (0.0007) [2023-10-12 23:16:29,945][44958] Updated weights for policy 0, policy_version 77040 (0.0007) [2023-10-12 23:16:30,315][44958] Updated weights for policy 0, policy_version 77050 (0.0008) [2023-10-12 23:16:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158171136. Throughput: 0: 1638.6, 1: 1653.7. Samples: 39551274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:16:31,443][43579] Avg episode reward: [(0, '282.730'), (1, '277.380')] [2023-10-12 23:16:33,246][44959] Updated weights for policy 1, policy_version 77410 (0.0008) [2023-10-12 23:16:33,619][44959] Updated weights for policy 1, policy_version 77420 (0.0009) [2023-10-12 23:16:33,989][44959] Updated weights for policy 1, policy_version 77430 (0.0010) [2023-10-12 23:16:34,353][44959] Updated weights for policy 1, policy_version 77440 (0.0010) [2023-10-12 23:16:34,423][44958] Updated weights for policy 0, policy_version 77060 (0.0009) [2023-10-12 23:16:34,801][44958] Updated weights for policy 0, policy_version 77070 (0.0010) [2023-10-12 23:16:35,168][44958] Updated weights for policy 0, policy_version 77080 (0.0008) [2023-10-12 23:16:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158236672. Throughput: 0: 1642.2, 1: 1649.0. Samples: 39561716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:16:36,443][43579] Avg episode reward: [(0, '281.620'), (1, '275.170')] [2023-10-12 23:16:38,409][44959] Updated weights for policy 1, policy_version 77450 (0.0010) [2023-10-12 23:16:38,784][44959] Updated weights for policy 1, policy_version 77460 (0.0008) [2023-10-12 23:16:39,157][44959] Updated weights for policy 1, policy_version 77470 (0.0008) [2023-10-12 23:16:39,252][44958] Updated weights for policy 0, policy_version 77090 (0.0009) [2023-10-12 23:16:39,624][44958] Updated weights for policy 0, policy_version 77100 (0.0010) [2023-10-12 23:16:39,998][44958] Updated weights for policy 0, policy_version 77110 (0.0010) [2023-10-12 23:16:40,369][44958] Updated weights for policy 0, policy_version 77120 (0.0010) [2023-10-12 23:16:41,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158302208. Throughput: 0: 1638.8, 1: 1651.4. Samples: 39580722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:16:41,444][43579] Avg episode reward: [(0, '281.700'), (1, '272.280')] [2023-10-12 23:16:42,909][44959] Updated weights for policy 1, policy_version 77480 (0.0008) [2023-10-12 23:16:43,282][44959] Updated weights for policy 1, policy_version 77490 (0.0009) [2023-10-12 23:16:43,646][44959] Updated weights for policy 1, policy_version 77500 (0.0007) [2023-10-12 23:16:44,491][44958] Updated weights for policy 0, policy_version 77130 (0.0007) [2023-10-12 23:16:44,866][44958] Updated weights for policy 0, policy_version 77140 (0.0007) [2023-10-12 23:16:45,223][44958] Updated weights for policy 0, policy_version 77150 (0.0008) [2023-10-12 23:16:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 158367744. Throughput: 0: 1642.6, 1: 1651.7. Samples: 39600766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:16:46,443][43579] Avg episode reward: [(0, '282.160'), (1, '276.840')] [2023-10-12 23:16:47,923][44959] Updated weights for policy 1, policy_version 77510 (0.0008) [2023-10-12 23:16:48,290][44959] Updated weights for policy 1, policy_version 77520 (0.0008) [2023-10-12 23:16:48,663][44959] Updated weights for policy 1, policy_version 77530 (0.0007) [2023-10-12 23:16:49,726][44958] Updated weights for policy 0, policy_version 77160 (0.0010) [2023-10-12 23:16:50,095][44958] Updated weights for policy 0, policy_version 77170 (0.0010) [2023-10-12 23:16:50,469][44958] Updated weights for policy 0, policy_version 77180 (0.0008) [2023-10-12 23:16:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158433280. Throughput: 0: 1641.6, 1: 1638.7. Samples: 39610788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:16:51,443][43579] Avg episode reward: [(0, '284.240'), (1, '275.590')] [2023-10-12 23:16:52,600][44959] Updated weights for policy 1, policy_version 77540 (0.0009) [2023-10-12 23:16:52,967][44959] Updated weights for policy 1, policy_version 77550 (0.0008) [2023-10-12 23:16:53,340][44959] Updated weights for policy 1, policy_version 77560 (0.0010) [2023-10-12 23:16:54,464][44958] Updated weights for policy 0, policy_version 77190 (0.0008) [2023-10-12 23:16:54,829][44958] Updated weights for policy 0, policy_version 77200 (0.0008) [2023-10-12 23:16:55,208][44958] Updated weights for policy 0, policy_version 77210 (0.0009) [2023-10-12 23:16:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158498816. Throughput: 0: 1641.0, 1: 1649.4. Samples: 39630370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:16:56,443][43579] Avg episode reward: [(0, '284.750'), (1, '281.080')] [2023-10-12 23:16:57,702][44959] Updated weights for policy 1, policy_version 77570 (0.0010) [2023-10-12 23:16:58,062][44959] Updated weights for policy 1, policy_version 77580 (0.0010) [2023-10-12 23:16:58,436][44959] Updated weights for policy 1, policy_version 77590 (0.0008) [2023-10-12 23:16:58,808][44959] Updated weights for policy 1, policy_version 77600 (0.0008) [2023-10-12 23:16:59,370][44958] Updated weights for policy 0, policy_version 77220 (0.0010) [2023-10-12 23:16:59,742][44958] Updated weights for policy 0, policy_version 77230 (0.0010) [2023-10-12 23:17:00,115][44958] Updated weights for policy 0, policy_version 77240 (0.0011) [2023-10-12 23:17:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 158564352. Throughput: 0: 1649.4, 1: 1646.3. Samples: 39650378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:17:01,443][43579] Avg episode reward: [(0, '286.740'), (1, '278.690')] [2023-10-12 23:17:03,021][44959] Updated weights for policy 1, policy_version 77610 (0.0008) [2023-10-12 23:17:03,394][44959] Updated weights for policy 1, policy_version 77620 (0.0008) [2023-10-12 23:17:03,758][44959] Updated weights for policy 1, policy_version 77630 (0.0008) [2023-10-12 23:17:04,285][44958] Updated weights for policy 0, policy_version 77250 (0.0009) [2023-10-12 23:17:04,670][44958] Updated weights for policy 0, policy_version 77260 (0.0011) [2023-10-12 23:17:05,047][44958] Updated weights for policy 0, policy_version 77270 (0.0010) [2023-10-12 23:17:05,422][44958] Updated weights for policy 0, policy_version 77280 (0.0009) [2023-10-12 23:17:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158629888. Throughput: 0: 1654.8, 1: 1637.4. Samples: 39660546. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:06,444][43579] Avg episode reward: [(0, '286.000'), (1, '283.480')] [2023-10-12 23:17:07,950][44959] Updated weights for policy 1, policy_version 77640 (0.0009) [2023-10-12 23:17:08,314][44959] Updated weights for policy 1, policy_version 77650 (0.0007) [2023-10-12 23:17:08,681][44959] Updated weights for policy 1, policy_version 77660 (0.0008) [2023-10-12 23:17:09,571][44958] Updated weights for policy 0, policy_version 77290 (0.0009) [2023-10-12 23:17:09,943][44958] Updated weights for policy 0, policy_version 77300 (0.0009) [2023-10-12 23:17:10,317][44958] Updated weights for policy 0, policy_version 77310 (0.0008) [2023-10-12 23:17:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158695424. Throughput: 0: 1647.6, 1: 1652.0. Samples: 39679856. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:11,443][43579] Avg episode reward: [(0, '282.050'), (1, '283.480')] [2023-10-12 23:17:12,878][44959] Updated weights for policy 1, policy_version 77670 (0.0009) [2023-10-12 23:17:13,249][44959] Updated weights for policy 1, policy_version 77680 (0.0008) [2023-10-12 23:17:13,620][44959] Updated weights for policy 1, policy_version 77690 (0.0007) [2023-10-12 23:17:14,620][44958] Updated weights for policy 0, policy_version 77320 (0.0009) [2023-10-12 23:17:15,002][44958] Updated weights for policy 0, policy_version 77330 (0.0007) [2023-10-12 23:17:15,377][44958] Updated weights for policy 0, policy_version 77340 (0.0009) [2023-10-12 23:17:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158760960. Throughput: 0: 1645.5, 1: 1653.2. Samples: 39699716. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:16,444][43579] Avg episode reward: [(0, '281.290'), (1, '282.670')] [2023-10-12 23:17:17,832][44959] Updated weights for policy 1, policy_version 77700 (0.0007) [2023-10-12 23:17:18,208][44959] Updated weights for policy 1, policy_version 77710 (0.0010) [2023-10-12 23:17:18,576][44959] Updated weights for policy 1, policy_version 77720 (0.0009) [2023-10-12 23:17:19,425][44958] Updated weights for policy 0, policy_version 77350 (0.0009) [2023-10-12 23:17:19,796][44958] Updated weights for policy 0, policy_version 77360 (0.0008) [2023-10-12 23:17:20,171][44958] Updated weights for policy 0, policy_version 77370 (0.0010) [2023-10-12 23:17:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158826496. Throughput: 0: 1650.8, 1: 1644.5. Samples: 39710004. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:21,444][43579] Avg episode reward: [(0, '273.550'), (1, '284.110')] [2023-10-12 23:17:22,734][44959] Updated weights for policy 1, policy_version 77730 (0.0009) [2023-10-12 23:17:23,096][44959] Updated weights for policy 1, policy_version 77740 (0.0007) [2023-10-12 23:17:23,465][44959] Updated weights for policy 1, policy_version 77750 (0.0007) [2023-10-12 23:17:23,830][44959] Updated weights for policy 1, policy_version 77760 (0.0007) [2023-10-12 23:17:24,365][44958] Updated weights for policy 0, policy_version 77380 (0.0007) [2023-10-12 23:17:24,746][44958] Updated weights for policy 0, policy_version 77390 (0.0010) [2023-10-12 23:17:25,108][44958] Updated weights for policy 0, policy_version 77400 (0.0008) [2023-10-12 23:17:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 158892032. Throughput: 0: 1645.1, 1: 1655.5. Samples: 39729250. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:26,443][43579] Avg episode reward: [(0, '265.800'), (1, '284.080')] [2023-10-12 23:17:27,941][44959] Updated weights for policy 1, policy_version 77770 (0.0008) [2023-10-12 23:17:28,317][44959] Updated weights for policy 1, policy_version 77780 (0.0008) [2023-10-12 23:17:28,687][44959] Updated weights for policy 1, policy_version 77790 (0.0008) [2023-10-12 23:17:29,184][44958] Updated weights for policy 0, policy_version 77410 (0.0008) [2023-10-12 23:17:29,556][44958] Updated weights for policy 0, policy_version 77420 (0.0011) [2023-10-12 23:17:29,933][44958] Updated weights for policy 0, policy_version 77430 (0.0007) [2023-10-12 23:17:30,305][44958] Updated weights for policy 0, policy_version 77440 (0.0008) [2023-10-12 23:17:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 158957568. Throughput: 0: 1650.2, 1: 1650.6. Samples: 39749304. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:31,444][43579] Avg episode reward: [(0, '264.230'), (1, '285.980')] [2023-10-12 23:17:32,929][44959] Updated weights for policy 1, policy_version 77800 (0.0007) [2023-10-12 23:17:33,291][44959] Updated weights for policy 1, policy_version 77810 (0.0009) [2023-10-12 23:17:33,665][44959] Updated weights for policy 1, policy_version 77820 (0.0009) [2023-10-12 23:17:34,524][44958] Updated weights for policy 0, policy_version 77450 (0.0007) [2023-10-12 23:17:34,889][44958] Updated weights for policy 0, policy_version 77460 (0.0010) [2023-10-12 23:17:35,249][44958] Updated weights for policy 0, policy_version 77470 (0.0010) [2023-10-12 23:17:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159023104. Throughput: 0: 1650.6, 1: 1650.3. Samples: 39759328. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:36,444][43579] Avg episode reward: [(0, '262.310'), (1, '283.780')] [2023-10-12 23:17:37,772][44959] Updated weights for policy 1, policy_version 77830 (0.0008) [2023-10-12 23:17:38,145][44959] Updated weights for policy 1, policy_version 77840 (0.0008) [2023-10-12 23:17:38,515][44959] Updated weights for policy 1, policy_version 77850 (0.0010) [2023-10-12 23:17:39,359][44958] Updated weights for policy 0, policy_version 77480 (0.0010) [2023-10-12 23:17:39,731][44958] Updated weights for policy 0, policy_version 77490 (0.0009) [2023-10-12 23:17:40,095][44958] Updated weights for policy 0, policy_version 77500 (0.0008) [2023-10-12 23:17:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159088640. Throughput: 0: 1644.9, 1: 1650.4. Samples: 39778662. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:41,444][43579] Avg episode reward: [(0, '267.790'), (1, '283.400')] [2023-10-12 23:17:42,555][44959] Updated weights for policy 1, policy_version 77860 (0.0007) [2023-10-12 23:17:42,933][44959] Updated weights for policy 1, policy_version 77870 (0.0008) [2023-10-12 23:17:43,298][44959] Updated weights for policy 1, policy_version 77880 (0.0007) [2023-10-12 23:17:44,127][44958] Updated weights for policy 0, policy_version 77510 (0.0008) [2023-10-12 23:17:44,484][44958] Updated weights for policy 0, policy_version 77520 (0.0010) [2023-10-12 23:17:44,858][44958] Updated weights for policy 0, policy_version 77530 (0.0009) [2023-10-12 23:17:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159154176. Throughput: 0: 1644.4, 1: 1660.0. Samples: 39799076. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:46,443][43579] Avg episode reward: [(0, '266.300'), (1, '285.560')] [2023-10-12 23:17:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000077536_79396864.pth... [2023-10-12 23:17:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000077888_79757312.pth... [2023-10-12 23:17:46,480][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000076000_77824000.pth [2023-10-12 23:17:46,487][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000076352_78184448.pth [2023-10-12 23:17:47,506][44959] Updated weights for policy 1, policy_version 77890 (0.0009) [2023-10-12 23:17:47,887][44959] Updated weights for policy 1, policy_version 77900 (0.0008) [2023-10-12 23:17:48,247][44959] Updated weights for policy 1, policy_version 77910 (0.0009) [2023-10-12 23:17:48,614][44959] Updated weights for policy 1, policy_version 77920 (0.0009) [2023-10-12 23:17:49,023][44958] Updated weights for policy 0, policy_version 77540 (0.0008) [2023-10-12 23:17:49,396][44958] Updated weights for policy 0, policy_version 77550 (0.0010) [2023-10-12 23:17:49,770][44958] Updated weights for policy 0, policy_version 77560 (0.0007) [2023-10-12 23:17:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159219712. Throughput: 0: 1636.4, 1: 1658.9. Samples: 39808834. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-12 23:17:51,444][43579] Avg episode reward: [(0, '272.640'), (1, '281.640')] [2023-10-12 23:17:52,752][44959] Updated weights for policy 1, policy_version 77930 (0.0008) [2023-10-12 23:17:53,118][44959] Updated weights for policy 1, policy_version 77940 (0.0008) [2023-10-12 23:17:53,478][44959] Updated weights for policy 1, policy_version 77950 (0.0010) [2023-10-12 23:17:54,123][44958] Updated weights for policy 0, policy_version 77570 (0.0009) [2023-10-12 23:17:54,496][44958] Updated weights for policy 0, policy_version 77580 (0.0009) [2023-10-12 23:17:54,864][44958] Updated weights for policy 0, policy_version 77590 (0.0008) [2023-10-12 23:17:55,234][44958] Updated weights for policy 0, policy_version 77600 (0.0012) [2023-10-12 23:17:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159285248. Throughput: 0: 1640.6, 1: 1656.7. Samples: 39828234. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:17:56,443][43579] Avg episode reward: [(0, '276.750'), (1, '285.690')] [2023-10-12 23:17:57,828][44959] Updated weights for policy 1, policy_version 77960 (0.0008) [2023-10-12 23:17:58,191][44959] Updated weights for policy 1, policy_version 77970 (0.0009) [2023-10-12 23:17:58,566][44959] Updated weights for policy 1, policy_version 77980 (0.0011) [2023-10-12 23:17:59,555][44958] Updated weights for policy 0, policy_version 77610 (0.0009) [2023-10-12 23:17:59,924][44958] Updated weights for policy 0, policy_version 77620 (0.0011) [2023-10-12 23:18:00,296][44958] Updated weights for policy 0, policy_version 77630 (0.0007) [2023-10-12 23:18:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159350784. Throughput: 0: 1637.4, 1: 1657.8. Samples: 39847998. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:18:01,443][43579] Avg episode reward: [(0, '277.180'), (1, '287.020')] [2023-10-12 23:18:02,535][44959] Updated weights for policy 1, policy_version 77990 (0.0008) [2023-10-12 23:18:02,904][44959] Updated weights for policy 1, policy_version 78000 (0.0009) [2023-10-12 23:18:03,278][44959] Updated weights for policy 1, policy_version 78010 (0.0008) [2023-10-12 23:18:04,582][44958] Updated weights for policy 0, policy_version 77640 (0.0008) [2023-10-12 23:18:04,965][44958] Updated weights for policy 0, policy_version 77650 (0.0008) [2023-10-12 23:18:05,339][44958] Updated weights for policy 0, policy_version 77660 (0.0009) [2023-10-12 23:18:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159416320. Throughput: 0: 1632.5, 1: 1656.2. Samples: 39857994. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:18:06,443][43579] Avg episode reward: [(0, '279.960'), (1, '283.600')] [2023-10-12 23:18:07,506][44959] Updated weights for policy 1, policy_version 78020 (0.0007) [2023-10-12 23:18:07,876][44959] Updated weights for policy 1, policy_version 78030 (0.0009) [2023-10-12 23:18:08,247][44959] Updated weights for policy 1, policy_version 78040 (0.0008) [2023-10-12 23:18:09,562][44958] Updated weights for policy 0, policy_version 77670 (0.0009) [2023-10-12 23:18:09,922][44958] Updated weights for policy 0, policy_version 77680 (0.0009) [2023-10-12 23:18:10,293][44958] Updated weights for policy 0, policy_version 77690 (0.0008) [2023-10-12 23:18:11,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 159481856. Throughput: 0: 1639.9, 1: 1660.9. Samples: 39877788. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:18:11,444][43579] Avg episode reward: [(0, '277.140'), (1, '280.910')] [2023-10-12 23:18:12,268][44959] Updated weights for policy 1, policy_version 78050 (0.0008) [2023-10-12 23:18:12,630][44959] Updated weights for policy 1, policy_version 78060 (0.0007) [2023-10-12 23:18:12,991][44959] Updated weights for policy 1, policy_version 78070 (0.0009) [2023-10-12 23:18:13,356][44959] Updated weights for policy 1, policy_version 78080 (0.0007) [2023-10-12 23:18:14,556][44958] Updated weights for policy 0, policy_version 77700 (0.0008) [2023-10-12 23:18:14,934][44958] Updated weights for policy 0, policy_version 77710 (0.0010) [2023-10-12 23:18:15,305][44958] Updated weights for policy 0, policy_version 77720 (0.0008) [2023-10-12 23:18:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 159547392. Throughput: 0: 1632.0, 1: 1667.2. Samples: 39897768. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:18:16,443][43579] Avg episode reward: [(0, '271.840'), (1, '279.500')] [2023-10-12 23:18:17,446][44959] Updated weights for policy 1, policy_version 78090 (0.0010) [2023-10-12 23:18:17,816][44959] Updated weights for policy 1, policy_version 78100 (0.0010) [2023-10-12 23:18:18,181][44959] Updated weights for policy 1, policy_version 78110 (0.0007) [2023-10-12 23:18:19,296][44958] Updated weights for policy 0, policy_version 77730 (0.0009) [2023-10-12 23:18:19,669][44958] Updated weights for policy 0, policy_version 77740 (0.0009) [2023-10-12 23:18:20,046][44958] Updated weights for policy 0, policy_version 77750 (0.0007) [2023-10-12 23:18:20,408][44958] Updated weights for policy 0, policy_version 77760 (0.0009) [2023-10-12 23:18:21,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159612928. Throughput: 0: 1637.4, 1: 1668.7. Samples: 39908100. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:18:21,443][43579] Avg episode reward: [(0, '272.100'), (1, '280.430')] [2023-10-12 23:18:22,293][44959] Updated weights for policy 1, policy_version 78120 (0.0007) [2023-10-12 23:18:22,660][44959] Updated weights for policy 1, policy_version 78130 (0.0007) [2023-10-12 23:18:23,032][44959] Updated weights for policy 1, policy_version 78140 (0.0009) [2023-10-12 23:18:24,464][44958] Updated weights for policy 0, policy_version 77770 (0.0009) [2023-10-12 23:18:24,834][44958] Updated weights for policy 0, policy_version 77780 (0.0009) [2023-10-12 23:18:25,213][44958] Updated weights for policy 0, policy_version 77790 (0.0009) [2023-10-12 23:18:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159678464. Throughput: 0: 1639.5, 1: 1667.6. Samples: 39927480. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:18:26,443][43579] Avg episode reward: [(0, '267.870'), (1, '277.270')] [2023-10-12 23:18:26,954][44959] Updated weights for policy 1, policy_version 78150 (0.0011) [2023-10-12 23:18:27,316][44959] Updated weights for policy 1, policy_version 78160 (0.0010) [2023-10-12 23:18:27,687][44959] Updated weights for policy 1, policy_version 78170 (0.0009) [2023-10-12 23:18:29,353][44958] Updated weights for policy 0, policy_version 77800 (0.0007) [2023-10-12 23:18:29,726][44958] Updated weights for policy 0, policy_version 77810 (0.0008) [2023-10-12 23:18:30,087][44958] Updated weights for policy 0, policy_version 77820 (0.0009) [2023-10-12 23:18:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159744000. Throughput: 0: 1641.3, 1: 1659.6. Samples: 39947614. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:18:31,444][43579] Avg episode reward: [(0, '265.860'), (1, '278.570')] [2023-10-12 23:18:31,960][44959] Updated weights for policy 1, policy_version 78180 (0.0008) [2023-10-12 23:18:32,326][44959] Updated weights for policy 1, policy_version 78190 (0.0008) [2023-10-12 23:18:32,697][44959] Updated weights for policy 1, policy_version 78200 (0.0008) [2023-10-12 23:18:34,381][44958] Updated weights for policy 0, policy_version 77830 (0.0007) [2023-10-12 23:18:34,749][44958] Updated weights for policy 0, policy_version 77840 (0.0007) [2023-10-12 23:18:35,121][44958] Updated weights for policy 0, policy_version 77850 (0.0009) [2023-10-12 23:18:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 159809536. Throughput: 0: 1646.4, 1: 1659.1. Samples: 39957582. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:18:36,444][43579] Avg episode reward: [(0, '261.360'), (1, '276.750')] [2023-10-12 23:18:36,722][44959] Updated weights for policy 1, policy_version 78210 (0.0009) [2023-10-12 23:18:37,106][44959] Updated weights for policy 1, policy_version 78220 (0.0007) [2023-10-12 23:18:37,477][44959] Updated weights for policy 1, policy_version 78230 (0.0007) [2023-10-12 23:18:37,842][44959] Updated weights for policy 1, policy_version 78240 (0.0009) [2023-10-12 23:18:39,141][44958] Updated weights for policy 0, policy_version 77860 (0.0010) [2023-10-12 23:18:39,510][44958] Updated weights for policy 0, policy_version 77870 (0.0008) [2023-10-12 23:18:39,878][44958] Updated weights for policy 0, policy_version 77880 (0.0007) [2023-10-12 23:18:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159875072. Throughput: 0: 1640.4, 1: 1660.7. Samples: 39976784. Policy #0 lag: (min: 18.0, avg: 18.0, max: 20.0) [2023-10-12 23:18:41,444][43579] Avg episode reward: [(0, '264.580'), (1, '282.560')] [2023-10-12 23:18:42,056][44959] Updated weights for policy 1, policy_version 78250 (0.0011) [2023-10-12 23:18:42,427][44959] Updated weights for policy 1, policy_version 78260 (0.0012) [2023-10-12 23:18:42,802][44959] Updated weights for policy 1, policy_version 78270 (0.0011) [2023-10-12 23:18:44,149][44958] Updated weights for policy 0, policy_version 77890 (0.0008) [2023-10-12 23:18:44,537][44958] Updated weights for policy 0, policy_version 77900 (0.0008) [2023-10-12 23:18:44,906][44958] Updated weights for policy 0, policy_version 77910 (0.0007) [2023-10-12 23:18:45,274][44958] Updated weights for policy 0, policy_version 77920 (0.0007) [2023-10-12 23:18:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 159940608. Throughput: 0: 1652.6, 1: 1655.6. Samples: 39996868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:18:46,443][43579] Avg episode reward: [(0, '266.750'), (1, '283.460')] [2023-10-12 23:18:47,047][44959] Updated weights for policy 1, policy_version 78280 (0.0010) [2023-10-12 23:18:47,413][44959] Updated weights for policy 1, policy_version 78290 (0.0009) [2023-10-12 23:18:47,785][44959] Updated weights for policy 1, policy_version 78300 (0.0009) [2023-10-12 23:18:49,544][44958] Updated weights for policy 0, policy_version 77930 (0.0009) [2023-10-12 23:18:49,910][44958] Updated weights for policy 0, policy_version 77940 (0.0007) [2023-10-12 23:18:50,293][44958] Updated weights for policy 0, policy_version 77950 (0.0008) [2023-10-12 23:18:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160006144. Throughput: 0: 1645.9, 1: 1646.8. Samples: 40006166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:18:51,443][43579] Avg episode reward: [(0, '264.340'), (1, '283.290')] [2023-10-12 23:18:52,172][44959] Updated weights for policy 1, policy_version 78310 (0.0009) [2023-10-12 23:18:52,542][44959] Updated weights for policy 1, policy_version 78320 (0.0008) [2023-10-12 23:18:52,908][44959] Updated weights for policy 1, policy_version 78330 (0.0010) [2023-10-12 23:18:54,490][44958] Updated weights for policy 0, policy_version 77960 (0.0008) [2023-10-12 23:18:54,870][44958] Updated weights for policy 0, policy_version 77970 (0.0009) [2023-10-12 23:18:55,248][44958] Updated weights for policy 0, policy_version 77980 (0.0009) [2023-10-12 23:18:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160071680. Throughput: 0: 1639.0, 1: 1643.5. Samples: 40025500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:18:56,444][43579] Avg episode reward: [(0, '272.280'), (1, '287.940')] [2023-10-12 23:18:57,156][44959] Updated weights for policy 1, policy_version 78340 (0.0008) [2023-10-12 23:18:57,523][44959] Updated weights for policy 1, policy_version 78350 (0.0008) [2023-10-12 23:18:57,886][44959] Updated weights for policy 1, policy_version 78360 (0.0009) [2023-10-12 23:18:59,362][44958] Updated weights for policy 0, policy_version 77990 (0.0009) [2023-10-12 23:18:59,730][44958] Updated weights for policy 0, policy_version 78000 (0.0007) [2023-10-12 23:19:00,101][44958] Updated weights for policy 0, policy_version 78010 (0.0007) [2023-10-12 23:19:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 160137216. Throughput: 0: 1643.2, 1: 1638.3. Samples: 40045436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:01,444][43579] Avg episode reward: [(0, '270.090'), (1, '289.960')] [2023-10-12 23:19:02,027][44959] Updated weights for policy 1, policy_version 78370 (0.0008) [2023-10-12 23:19:02,397][44959] Updated weights for policy 1, policy_version 78380 (0.0007) [2023-10-12 23:19:02,764][44959] Updated weights for policy 1, policy_version 78390 (0.0007) [2023-10-12 23:19:03,127][44959] Updated weights for policy 1, policy_version 78400 (0.0008) [2023-10-12 23:19:04,186][44958] Updated weights for policy 0, policy_version 78020 (0.0009) [2023-10-12 23:19:04,569][44958] Updated weights for policy 0, policy_version 78030 (0.0010) [2023-10-12 23:19:04,936][44958] Updated weights for policy 0, policy_version 78040 (0.0007) [2023-10-12 23:19:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160202752. Throughput: 0: 1638.1, 1: 1635.9. Samples: 40055430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:06,444][43579] Avg episode reward: [(0, '279.560'), (1, '286.710')] [2023-10-12 23:19:07,311][44959] Updated weights for policy 1, policy_version 78410 (0.0007) [2023-10-12 23:19:07,682][44959] Updated weights for policy 1, policy_version 78420 (0.0007) [2023-10-12 23:19:08,043][44959] Updated weights for policy 1, policy_version 78430 (0.0008) [2023-10-12 23:19:09,122][44958] Updated weights for policy 0, policy_version 78050 (0.0009) [2023-10-12 23:19:09,497][44958] Updated weights for policy 0, policy_version 78060 (0.0007) [2023-10-12 23:19:09,864][44958] Updated weights for policy 0, policy_version 78070 (0.0007) [2023-10-12 23:19:10,237][44958] Updated weights for policy 0, policy_version 78080 (0.0008) [2023-10-12 23:19:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160268288. Throughput: 0: 1639.4, 1: 1639.7. Samples: 40075040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:11,443][43579] Avg episode reward: [(0, '271.040'), (1, '277.920')] [2023-10-12 23:19:12,127][44959] Updated weights for policy 1, policy_version 78440 (0.0010) [2023-10-12 23:19:12,494][44959] Updated weights for policy 1, policy_version 78450 (0.0010) [2023-10-12 23:19:12,865][44959] Updated weights for policy 1, policy_version 78460 (0.0011) [2023-10-12 23:19:14,422][44958] Updated weights for policy 0, policy_version 78090 (0.0007) [2023-10-12 23:19:14,787][44958] Updated weights for policy 0, policy_version 78100 (0.0008) [2023-10-12 23:19:15,158][44958] Updated weights for policy 0, policy_version 78110 (0.0007) [2023-10-12 23:19:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160333824. Throughput: 0: 1634.5, 1: 1640.8. Samples: 40095004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:16,443][43579] Avg episode reward: [(0, '268.360'), (1, '281.370')] [2023-10-12 23:19:16,947][44959] Updated weights for policy 1, policy_version 78470 (0.0008) [2023-10-12 23:19:17,319][44959] Updated weights for policy 1, policy_version 78480 (0.0007) [2023-10-12 23:19:17,691][44959] Updated weights for policy 1, policy_version 78490 (0.0007) [2023-10-12 23:19:19,398][44958] Updated weights for policy 0, policy_version 78120 (0.0007) [2023-10-12 23:19:19,765][44958] Updated weights for policy 0, policy_version 78130 (0.0007) [2023-10-12 23:19:20,139][44958] Updated weights for policy 0, policy_version 78140 (0.0009) [2023-10-12 23:19:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160399360. Throughput: 0: 1634.1, 1: 1643.3. Samples: 40105064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:21,443][43579] Avg episode reward: [(0, '270.290'), (1, '278.650')] [2023-10-12 23:19:21,860][44959] Updated weights for policy 1, policy_version 78500 (0.0008) [2023-10-12 23:19:22,242][44959] Updated weights for policy 1, policy_version 78510 (0.0008) [2023-10-12 23:19:22,612][44959] Updated weights for policy 1, policy_version 78520 (0.0007) [2023-10-12 23:19:24,601][44958] Updated weights for policy 0, policy_version 78150 (0.0007) [2023-10-12 23:19:24,973][44958] Updated weights for policy 0, policy_version 78160 (0.0008) [2023-10-12 23:19:25,353][44958] Updated weights for policy 0, policy_version 78170 (0.0007) [2023-10-12 23:19:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160464896. Throughput: 0: 1638.1, 1: 1640.8. Samples: 40124334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:26,444][43579] Avg episode reward: [(0, '266.740'), (1, '272.900')] [2023-10-12 23:19:26,870][44959] Updated weights for policy 1, policy_version 78530 (0.0008) [2023-10-12 23:19:27,236][44959] Updated weights for policy 1, policy_version 78540 (0.0009) [2023-10-12 23:19:27,601][44959] Updated weights for policy 1, policy_version 78550 (0.0008) [2023-10-12 23:19:27,965][44959] Updated weights for policy 1, policy_version 78560 (0.0009) [2023-10-12 23:19:29,164][44958] Updated weights for policy 0, policy_version 78180 (0.0009) [2023-10-12 23:19:29,556][44958] Updated weights for policy 0, policy_version 78190 (0.0008) [2023-10-12 23:19:29,929][44958] Updated weights for policy 0, policy_version 78200 (0.0007) [2023-10-12 23:19:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160530432. Throughput: 0: 1627.9, 1: 1641.8. Samples: 40144004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:31,444][43579] Avg episode reward: [(0, '263.150'), (1, '275.340')] [2023-10-12 23:19:32,011][44959] Updated weights for policy 1, policy_version 78570 (0.0009) [2023-10-12 23:19:32,389][44959] Updated weights for policy 1, policy_version 78580 (0.0009) [2023-10-12 23:19:32,751][44959] Updated weights for policy 1, policy_version 78590 (0.0009) [2023-10-12 23:19:34,097][44958] Updated weights for policy 0, policy_version 78210 (0.0007) [2023-10-12 23:19:34,467][44958] Updated weights for policy 0, policy_version 78220 (0.0007) [2023-10-12 23:19:34,845][44958] Updated weights for policy 0, policy_version 78230 (0.0008) [2023-10-12 23:19:35,212][44958] Updated weights for policy 0, policy_version 78240 (0.0009) [2023-10-12 23:19:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160595968. Throughput: 0: 1635.6, 1: 1651.7. Samples: 40154092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:36,443][43579] Avg episode reward: [(0, '268.960'), (1, '276.080')] [2023-10-12 23:19:36,859][44959] Updated weights for policy 1, policy_version 78600 (0.0008) [2023-10-12 23:19:37,225][44959] Updated weights for policy 1, policy_version 78610 (0.0009) [2023-10-12 23:19:37,601][44959] Updated weights for policy 1, policy_version 78620 (0.0007) [2023-10-12 23:19:39,284][44958] Updated weights for policy 0, policy_version 78250 (0.0009) [2023-10-12 23:19:39,661][44958] Updated weights for policy 0, policy_version 78260 (0.0009) [2023-10-12 23:19:40,035][44958] Updated weights for policy 0, policy_version 78270 (0.0009) [2023-10-12 23:19:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160661504. Throughput: 0: 1634.8, 1: 1651.8. Samples: 40173394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:41,443][43579] Avg episode reward: [(0, '279.290'), (1, '282.570')] [2023-10-12 23:19:41,719][44959] Updated weights for policy 1, policy_version 78630 (0.0008) [2023-10-12 23:19:42,091][44959] Updated weights for policy 1, policy_version 78640 (0.0009) [2023-10-12 23:19:42,457][44959] Updated weights for policy 1, policy_version 78650 (0.0009) [2023-10-12 23:19:44,407][44958] Updated weights for policy 0, policy_version 78280 (0.0008) [2023-10-12 23:19:44,787][44958] Updated weights for policy 0, policy_version 78290 (0.0008) [2023-10-12 23:19:45,160][44958] Updated weights for policy 0, policy_version 78300 (0.0007) [2023-10-12 23:19:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160727040. Throughput: 0: 1637.9, 1: 1652.9. Samples: 40193522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:46,443][43579] Avg episode reward: [(0, '279.270'), (1, '276.280')] [2023-10-12 23:19:46,449][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000078304_80183296.pth... [2023-10-12 23:19:46,485][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000076768_78610432.pth [2023-10-12 23:19:46,668][44959] Updated weights for policy 1, policy_version 78660 (0.0009) [2023-10-12 23:19:47,040][44959] Updated weights for policy 1, policy_version 78670 (0.0009) [2023-10-12 23:19:47,413][44959] Updated weights for policy 1, policy_version 78680 (0.0008) [2023-10-12 23:19:47,708][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000078688_80576512.pth... [2023-10-12 23:19:47,750][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000077120_78970880.pth [2023-10-12 23:19:49,159][44958] Updated weights for policy 0, policy_version 78310 (0.0007) [2023-10-12 23:19:49,528][44958] Updated weights for policy 0, policy_version 78320 (0.0009) [2023-10-12 23:19:49,907][44958] Updated weights for policy 0, policy_version 78330 (0.0008) [2023-10-12 23:19:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160792576. Throughput: 0: 1636.1, 1: 1655.3. Samples: 40203542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:51,443][43579] Avg episode reward: [(0, '272.560'), (1, '278.670')] [2023-10-12 23:19:51,597][44959] Updated weights for policy 1, policy_version 78690 (0.0008) [2023-10-12 23:19:51,973][44959] Updated weights for policy 1, policy_version 78700 (0.0007) [2023-10-12 23:19:52,337][44959] Updated weights for policy 1, policy_version 78710 (0.0008) [2023-10-12 23:19:52,703][44959] Updated weights for policy 1, policy_version 78720 (0.0009) [2023-10-12 23:19:53,979][44958] Updated weights for policy 0, policy_version 78340 (0.0009) [2023-10-12 23:19:54,348][44958] Updated weights for policy 0, policy_version 78350 (0.0008) [2023-10-12 23:19:54,716][44958] Updated weights for policy 0, policy_version 78360 (0.0008) [2023-10-12 23:19:56,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160858112. Throughput: 0: 1637.1, 1: 1654.0. Samples: 40223138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:19:56,443][43579] Avg episode reward: [(0, '277.850'), (1, '279.600')] [2023-10-12 23:19:56,901][44959] Updated weights for policy 1, policy_version 78730 (0.0008) [2023-10-12 23:19:57,274][44959] Updated weights for policy 1, policy_version 78740 (0.0008) [2023-10-12 23:19:57,640][44959] Updated weights for policy 1, policy_version 78750 (0.0011) [2023-10-12 23:19:58,991][44958] Updated weights for policy 0, policy_version 78370 (0.0008) [2023-10-12 23:19:59,359][44958] Updated weights for policy 0, policy_version 78380 (0.0008) [2023-10-12 23:19:59,739][44958] Updated weights for policy 0, policy_version 78390 (0.0008) [2023-10-12 23:20:00,115][44958] Updated weights for policy 0, policy_version 78400 (0.0008) [2023-10-12 23:20:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160923648. Throughput: 0: 1644.3, 1: 1654.1. Samples: 40243430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:20:01,444][43579] Avg episode reward: [(0, '276.740'), (1, '280.590')] [2023-10-12 23:20:01,764][44959] Updated weights for policy 1, policy_version 78760 (0.0008) [2023-10-12 23:20:02,137][44959] Updated weights for policy 1, policy_version 78770 (0.0007) [2023-10-12 23:20:02,500][44959] Updated weights for policy 1, policy_version 78780 (0.0010) [2023-10-12 23:20:04,269][44958] Updated weights for policy 0, policy_version 78410 (0.0009) [2023-10-12 23:20:04,642][44958] Updated weights for policy 0, policy_version 78420 (0.0008) [2023-10-12 23:20:05,022][44958] Updated weights for policy 0, policy_version 78430 (0.0010) [2023-10-12 23:20:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 160989184. Throughput: 0: 1639.5, 1: 1653.4. Samples: 40253244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:20:06,444][43579] Avg episode reward: [(0, '273.590'), (1, '283.190')] [2023-10-12 23:20:06,946][44959] Updated weights for policy 1, policy_version 78790 (0.0008) [2023-10-12 23:20:07,343][44959] Updated weights for policy 1, policy_version 78800 (0.0007) [2023-10-12 23:20:07,718][44959] Updated weights for policy 1, policy_version 78810 (0.0009) [2023-10-12 23:20:09,210][44958] Updated weights for policy 0, policy_version 78440 (0.0010) [2023-10-12 23:20:09,584][44958] Updated weights for policy 0, policy_version 78450 (0.0010) [2023-10-12 23:20:09,958][44958] Updated weights for policy 0, policy_version 78460 (0.0008) [2023-10-12 23:20:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161054720. Throughput: 0: 1636.5, 1: 1649.8. Samples: 40272218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:20:11,444][43579] Avg episode reward: [(0, '272.860'), (1, '282.640')] [2023-10-12 23:20:11,714][44959] Updated weights for policy 1, policy_version 78820 (0.0009) [2023-10-12 23:20:12,085][44959] Updated weights for policy 1, policy_version 78830 (0.0009) [2023-10-12 23:20:12,455][44959] Updated weights for policy 1, policy_version 78840 (0.0007) [2023-10-12 23:20:14,345][44958] Updated weights for policy 0, policy_version 78470 (0.0009) [2023-10-12 23:20:14,705][44958] Updated weights for policy 0, policy_version 78480 (0.0008) [2023-10-12 23:20:15,077][44958] Updated weights for policy 0, policy_version 78490 (0.0010) [2023-10-12 23:20:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161120256. Throughput: 0: 1646.6, 1: 1656.4. Samples: 40292638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:20:16,443][43579] Avg episode reward: [(0, '272.370'), (1, '285.660')] [2023-10-12 23:20:16,554][44959] Updated weights for policy 1, policy_version 78850 (0.0008) [2023-10-12 23:20:16,928][44959] Updated weights for policy 1, policy_version 78860 (0.0010) [2023-10-12 23:20:17,289][44959] Updated weights for policy 1, policy_version 78870 (0.0011) [2023-10-12 23:20:17,657][44959] Updated weights for policy 1, policy_version 78880 (0.0011) [2023-10-12 23:20:19,102][44958] Updated weights for policy 0, policy_version 78500 (0.0008) [2023-10-12 23:20:19,478][44958] Updated weights for policy 0, policy_version 78510 (0.0009) [2023-10-12 23:20:19,851][44958] Updated weights for policy 0, policy_version 78520 (0.0008) [2023-10-12 23:20:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161185792. Throughput: 0: 1645.1, 1: 1656.0. Samples: 40302638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:20:21,443][43579] Avg episode reward: [(0, '268.140'), (1, '286.450')] [2023-10-12 23:20:21,956][44959] Updated weights for policy 1, policy_version 78890 (0.0008) [2023-10-12 23:20:22,319][44959] Updated weights for policy 1, policy_version 78900 (0.0007) [2023-10-12 23:20:22,692][44959] Updated weights for policy 1, policy_version 78910 (0.0009) [2023-10-12 23:20:23,862][44958] Updated weights for policy 0, policy_version 78530 (0.0008) [2023-10-12 23:20:24,237][44958] Updated weights for policy 0, policy_version 78540 (0.0011) [2023-10-12 23:20:24,607][44958] Updated weights for policy 0, policy_version 78550 (0.0008) [2023-10-12 23:20:24,979][44958] Updated weights for policy 0, policy_version 78560 (0.0007) [2023-10-12 23:20:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161251328. Throughput: 0: 1646.0, 1: 1656.1. Samples: 40321990. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:20:26,443][43579] Avg episode reward: [(0, '268.540'), (1, '286.410')] [2023-10-12 23:20:26,868][44959] Updated weights for policy 1, policy_version 78920 (0.0008) [2023-10-12 23:20:27,228][44959] Updated weights for policy 1, policy_version 78930 (0.0009) [2023-10-12 23:20:27,606][44959] Updated weights for policy 1, policy_version 78940 (0.0010) [2023-10-12 23:20:29,084][44958] Updated weights for policy 0, policy_version 78570 (0.0009) [2023-10-12 23:20:29,454][44958] Updated weights for policy 0, policy_version 78580 (0.0011) [2023-10-12 23:20:29,836][44958] Updated weights for policy 0, policy_version 78590 (0.0010) [2023-10-12 23:20:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161316864. Throughput: 0: 1653.4, 1: 1652.9. Samples: 40342308. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:20:31,443][43579] Avg episode reward: [(0, '265.330'), (1, '288.400')] [2023-10-12 23:20:31,681][44959] Updated weights for policy 1, policy_version 78950 (0.0009) [2023-10-12 23:20:32,055][44959] Updated weights for policy 1, policy_version 78960 (0.0009) [2023-10-12 23:20:32,417][44959] Updated weights for policy 1, policy_version 78970 (0.0009) [2023-10-12 23:20:34,043][44958] Updated weights for policy 0, policy_version 78600 (0.0009) [2023-10-12 23:20:34,411][44958] Updated weights for policy 0, policy_version 78610 (0.0007) [2023-10-12 23:20:34,782][44958] Updated weights for policy 0, policy_version 78620 (0.0009) [2023-10-12 23:20:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161382400. Throughput: 0: 1646.8, 1: 1648.7. Samples: 40351838. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:20:36,443][43579] Avg episode reward: [(0, '269.860'), (1, '284.200')] [2023-10-12 23:20:36,580][44959] Updated weights for policy 1, policy_version 78980 (0.0009) [2023-10-12 23:20:36,948][44959] Updated weights for policy 1, policy_version 78990 (0.0009) [2023-10-12 23:20:37,323][44959] Updated weights for policy 1, policy_version 79000 (0.0007) [2023-10-12 23:20:39,147][44958] Updated weights for policy 0, policy_version 78630 (0.0009) [2023-10-12 23:20:39,520][44958] Updated weights for policy 0, policy_version 78640 (0.0007) [2023-10-12 23:20:39,881][44958] Updated weights for policy 0, policy_version 78650 (0.0009) [2023-10-12 23:20:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161447936. Throughput: 0: 1640.5, 1: 1645.7. Samples: 40371020. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:20:41,443][43579] Avg episode reward: [(0, '264.470'), (1, '283.320')] [2023-10-12 23:20:41,608][44959] Updated weights for policy 1, policy_version 79010 (0.0008) [2023-10-12 23:20:41,978][44959] Updated weights for policy 1, policy_version 79020 (0.0008) [2023-10-12 23:20:42,353][44959] Updated weights for policy 1, policy_version 79030 (0.0007) [2023-10-12 23:20:42,715][44959] Updated weights for policy 1, policy_version 79040 (0.0008) [2023-10-12 23:20:43,913][44958] Updated weights for policy 0, policy_version 78660 (0.0008) [2023-10-12 23:20:44,294][44958] Updated weights for policy 0, policy_version 78670 (0.0010) [2023-10-12 23:20:44,669][44958] Updated weights for policy 0, policy_version 78680 (0.0010) [2023-10-12 23:20:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161513472. Throughput: 0: 1644.6, 1: 1644.5. Samples: 40391436. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:20:46,443][43579] Avg episode reward: [(0, '271.660'), (1, '281.660')] [2023-10-12 23:20:46,798][44959] Updated weights for policy 1, policy_version 79050 (0.0009) [2023-10-12 23:20:47,165][44959] Updated weights for policy 1, policy_version 79060 (0.0007) [2023-10-12 23:20:47,536][44959] Updated weights for policy 1, policy_version 79070 (0.0007) [2023-10-12 23:20:48,956][44958] Updated weights for policy 0, policy_version 78690 (0.0009) [2023-10-12 23:20:49,332][44958] Updated weights for policy 0, policy_version 78700 (0.0010) [2023-10-12 23:20:49,699][44958] Updated weights for policy 0, policy_version 78710 (0.0010) [2023-10-12 23:20:50,077][44958] Updated weights for policy 0, policy_version 78720 (0.0009) [2023-10-12 23:20:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161579008. Throughput: 0: 1639.8, 1: 1644.2. Samples: 40401022. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:20:51,443][43579] Avg episode reward: [(0, '275.830'), (1, '280.220')] [2023-10-12 23:20:51,623][44959] Updated weights for policy 1, policy_version 79080 (0.0009) [2023-10-12 23:20:52,000][44959] Updated weights for policy 1, policy_version 79090 (0.0008) [2023-10-12 23:20:52,369][44959] Updated weights for policy 1, policy_version 79100 (0.0008) [2023-10-12 23:20:54,335][44958] Updated weights for policy 0, policy_version 78730 (0.0011) [2023-10-12 23:20:54,706][44958] Updated weights for policy 0, policy_version 78740 (0.0009) [2023-10-12 23:20:55,071][44958] Updated weights for policy 0, policy_version 78750 (0.0007) [2023-10-12 23:20:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161644544. Throughput: 0: 1647.7, 1: 1653.8. Samples: 40420786. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:20:56,443][43579] Avg episode reward: [(0, '277.610'), (1, '278.380')] [2023-10-12 23:20:56,446][44959] Updated weights for policy 1, policy_version 79110 (0.0010) [2023-10-12 23:20:56,815][44959] Updated weights for policy 1, policy_version 79120 (0.0009) [2023-10-12 23:20:57,174][44959] Updated weights for policy 1, policy_version 79130 (0.0009) [2023-10-12 23:20:59,379][44958] Updated weights for policy 0, policy_version 78760 (0.0009) [2023-10-12 23:20:59,750][44958] Updated weights for policy 0, policy_version 78770 (0.0007) [2023-10-12 23:21:00,115][44958] Updated weights for policy 0, policy_version 78780 (0.0007) [2023-10-12 23:21:01,217][44959] Updated weights for policy 1, policy_version 79140 (0.0008) [2023-10-12 23:21:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161710080. Throughput: 0: 1639.0, 1: 1652.0. Samples: 40440732. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:21:01,443][43579] Avg episode reward: [(0, '278.820'), (1, '276.300')] [2023-10-12 23:21:01,590][44959] Updated weights for policy 1, policy_version 79150 (0.0008) [2023-10-12 23:21:01,952][44959] Updated weights for policy 1, policy_version 79160 (0.0010) [2023-10-12 23:21:04,191][44958] Updated weights for policy 0, policy_version 78790 (0.0009) [2023-10-12 23:21:04,560][44958] Updated weights for policy 0, policy_version 78800 (0.0008) [2023-10-12 23:21:04,931][44958] Updated weights for policy 0, policy_version 78810 (0.0007) [2023-10-12 23:21:06,003][44959] Updated weights for policy 1, policy_version 79170 (0.0009) [2023-10-12 23:21:06,377][44959] Updated weights for policy 1, policy_version 79180 (0.0008) [2023-10-12 23:21:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 161775616. Throughput: 0: 1636.2, 1: 1655.3. Samples: 40450754. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:21:06,443][43579] Avg episode reward: [(0, '272.370'), (1, '277.690')] [2023-10-12 23:21:06,740][44959] Updated weights for policy 1, policy_version 79190 (0.0009) [2023-10-12 23:21:07,112][44959] Updated weights for policy 1, policy_version 79200 (0.0009) [2023-10-12 23:21:09,169][44958] Updated weights for policy 0, policy_version 78820 (0.0009) [2023-10-12 23:21:09,539][44958] Updated weights for policy 0, policy_version 78830 (0.0009) [2023-10-12 23:21:09,922][44958] Updated weights for policy 0, policy_version 78840 (0.0008) [2023-10-12 23:21:11,231][44959] Updated weights for policy 1, policy_version 79210 (0.0007) [2023-10-12 23:21:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161841152. Throughput: 0: 1638.0, 1: 1657.1. Samples: 40470266. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:21:11,443][43579] Avg episode reward: [(0, '276.960'), (1, '277.000')] [2023-10-12 23:21:11,603][44959] Updated weights for policy 1, policy_version 79220 (0.0010) [2023-10-12 23:21:11,969][44959] Updated weights for policy 1, policy_version 79230 (0.0008) [2023-10-12 23:21:14,075][44958] Updated weights for policy 0, policy_version 78850 (0.0009) [2023-10-12 23:21:14,444][44958] Updated weights for policy 0, policy_version 78860 (0.0007) [2023-10-12 23:21:14,814][44958] Updated weights for policy 0, policy_version 78870 (0.0007) [2023-10-12 23:21:15,188][44958] Updated weights for policy 0, policy_version 78880 (0.0007) [2023-10-12 23:21:16,258][44959] Updated weights for policy 1, policy_version 79240 (0.0009) [2023-10-12 23:21:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161906688. Throughput: 0: 1631.9, 1: 1658.1. Samples: 40490358. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:21:16,443][43579] Avg episode reward: [(0, '271.840'), (1, '281.620')] [2023-10-12 23:21:16,625][44959] Updated weights for policy 1, policy_version 79250 (0.0008) [2023-10-12 23:21:17,001][44959] Updated weights for policy 1, policy_version 79260 (0.0010) [2023-10-12 23:21:19,046][44958] Updated weights for policy 0, policy_version 78890 (0.0011) [2023-10-12 23:21:19,415][44958] Updated weights for policy 0, policy_version 78900 (0.0011) [2023-10-12 23:21:19,786][44958] Updated weights for policy 0, policy_version 78910 (0.0011) [2023-10-12 23:21:21,079][44959] Updated weights for policy 1, policy_version 79270 (0.0009) [2023-10-12 23:21:21,438][44959] Updated weights for policy 1, policy_version 79280 (0.0010) [2023-10-12 23:21:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 161972224. Throughput: 0: 1632.0, 1: 1664.4. Samples: 40500178. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:21:21,444][43579] Avg episode reward: [(0, '269.250'), (1, '285.280')] [2023-10-12 23:21:21,806][44959] Updated weights for policy 1, policy_version 79290 (0.0009) [2023-10-12 23:21:24,388][44958] Updated weights for policy 0, policy_version 78920 (0.0010) [2023-10-12 23:21:24,758][44958] Updated weights for policy 0, policy_version 78930 (0.0008) [2023-10-12 23:21:25,125][44958] Updated weights for policy 0, policy_version 78940 (0.0008) [2023-10-12 23:21:26,043][44959] Updated weights for policy 1, policy_version 79300 (0.0008) [2023-10-12 23:21:26,402][44959] Updated weights for policy 1, policy_version 79310 (0.0008) [2023-10-12 23:21:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162037760. Throughput: 0: 1642.7, 1: 1666.5. Samples: 40519936. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:21:26,443][43579] Avg episode reward: [(0, '268.750'), (1, '284.020')] [2023-10-12 23:21:26,775][44959] Updated weights for policy 1, policy_version 79320 (0.0011) [2023-10-12 23:21:29,182][44958] Updated weights for policy 0, policy_version 78950 (0.0010) [2023-10-12 23:21:29,546][44958] Updated weights for policy 0, policy_version 78960 (0.0008) [2023-10-12 23:21:29,921][44958] Updated weights for policy 0, policy_version 78970 (0.0008) [2023-10-12 23:21:30,925][44959] Updated weights for policy 1, policy_version 79330 (0.0010) [2023-10-12 23:21:31,298][44959] Updated weights for policy 1, policy_version 79340 (0.0010) [2023-10-12 23:21:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162103296. Throughput: 0: 1632.6, 1: 1659.3. Samples: 40539574. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:21:31,443][43579] Avg episode reward: [(0, '268.170'), (1, '286.240')] [2023-10-12 23:21:31,664][44959] Updated weights for policy 1, policy_version 79350 (0.0010) [2023-10-12 23:21:32,026][44959] Updated weights for policy 1, policy_version 79360 (0.0010) [2023-10-12 23:21:34,069][44958] Updated weights for policy 0, policy_version 78980 (0.0007) [2023-10-12 23:21:34,441][44958] Updated weights for policy 0, policy_version 78990 (0.0007) [2023-10-12 23:21:34,820][44958] Updated weights for policy 0, policy_version 79000 (0.0009) [2023-10-12 23:21:36,029][44959] Updated weights for policy 1, policy_version 79370 (0.0007) [2023-10-12 23:21:36,397][44959] Updated weights for policy 1, policy_version 79380 (0.0008) [2023-10-12 23:21:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162168832. Throughput: 0: 1639.7, 1: 1662.1. Samples: 40549604. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:21:36,443][43579] Avg episode reward: [(0, '274.930'), (1, '288.810')] [2023-10-12 23:21:36,770][44959] Updated weights for policy 1, policy_version 79390 (0.0009) [2023-10-12 23:21:39,006][44958] Updated weights for policy 0, policy_version 79010 (0.0010) [2023-10-12 23:21:39,384][44958] Updated weights for policy 0, policy_version 79020 (0.0009) [2023-10-12 23:21:39,757][44958] Updated weights for policy 0, policy_version 79030 (0.0008) [2023-10-12 23:21:40,121][44958] Updated weights for policy 0, policy_version 79040 (0.0011) [2023-10-12 23:21:41,072][44959] Updated weights for policy 1, policy_version 79400 (0.0009) [2023-10-12 23:21:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162234368. Throughput: 0: 1634.2, 1: 1655.5. Samples: 40568822. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:21:41,444][43579] Avg episode reward: [(0, '276.390'), (1, '286.500')] [2023-10-12 23:21:41,448][44959] Updated weights for policy 1, policy_version 79410 (0.0010) [2023-10-12 23:21:41,813][44959] Updated weights for policy 1, policy_version 79420 (0.0010) [2023-10-12 23:21:44,412][44958] Updated weights for policy 0, policy_version 79050 (0.0007) [2023-10-12 23:21:44,787][44958] Updated weights for policy 0, policy_version 79060 (0.0007) [2023-10-12 23:21:45,160][44958] Updated weights for policy 0, policy_version 79070 (0.0009) [2023-10-12 23:21:45,860][44959] Updated weights for policy 1, policy_version 79430 (0.0009) [2023-10-12 23:21:46,225][44959] Updated weights for policy 1, policy_version 79440 (0.0008) [2023-10-12 23:21:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162299904. Throughput: 0: 1636.8, 1: 1646.4. Samples: 40588474. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:21:46,443][43579] Avg episode reward: [(0, '280.990'), (1, '289.030')] [2023-10-12 23:21:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000079072_80969728.pth... [2023-10-12 23:21:46,491][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000077536_79396864.pth [2023-10-12 23:21:46,602][44959] Updated weights for policy 1, policy_version 79450 (0.0009) [2023-10-12 23:21:46,814][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000079456_81362944.pth... [2023-10-12 23:21:46,843][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000077888_79757312.pth [2023-10-12 23:21:49,153][44958] Updated weights for policy 0, policy_version 79080 (0.0010) [2023-10-12 23:21:49,523][44958] Updated weights for policy 0, policy_version 79090 (0.0008) [2023-10-12 23:21:49,905][44958] Updated weights for policy 0, policy_version 79100 (0.0008) [2023-10-12 23:21:50,714][44959] Updated weights for policy 1, policy_version 79460 (0.0008) [2023-10-12 23:21:51,093][44959] Updated weights for policy 1, policy_version 79470 (0.0007) [2023-10-12 23:21:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162365440. Throughput: 0: 1632.7, 1: 1649.8. Samples: 40598468. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:21:51,443][43579] Avg episode reward: [(0, '281.830'), (1, '287.040')] [2023-10-12 23:21:51,450][44959] Updated weights for policy 1, policy_version 79480 (0.0009) [2023-10-12 23:21:54,107][44958] Updated weights for policy 0, policy_version 79110 (0.0010) [2023-10-12 23:21:54,476][44958] Updated weights for policy 0, policy_version 79120 (0.0007) [2023-10-12 23:21:54,857][44958] Updated weights for policy 0, policy_version 79130 (0.0008) [2023-10-12 23:21:55,507][44959] Updated weights for policy 1, policy_version 79490 (0.0011) [2023-10-12 23:21:55,873][44959] Updated weights for policy 1, policy_version 79500 (0.0008) [2023-10-12 23:21:56,241][44959] Updated weights for policy 1, policy_version 79510 (0.0007) [2023-10-12 23:21:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162430976. Throughput: 0: 1632.1, 1: 1645.5. Samples: 40617756. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:21:56,443][43579] Avg episode reward: [(0, '276.570'), (1, '281.020')] [2023-10-12 23:21:56,600][44959] Updated weights for policy 1, policy_version 79520 (0.0008) [2023-10-12 23:21:58,939][44958] Updated weights for policy 0, policy_version 79140 (0.0008) [2023-10-12 23:21:59,311][44958] Updated weights for policy 0, policy_version 79150 (0.0009) [2023-10-12 23:21:59,676][44958] Updated weights for policy 0, policy_version 79160 (0.0010) [2023-10-12 23:22:00,981][44959] Updated weights for policy 1, policy_version 79530 (0.0008) [2023-10-12 23:22:01,337][44959] Updated weights for policy 1, policy_version 79540 (0.0008) [2023-10-12 23:22:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162496512. Throughput: 0: 1638.4, 1: 1637.1. Samples: 40637754. Policy #0 lag: (min: 22.0, avg: 29.3, max: 54.0) [2023-10-12 23:22:01,443][43579] Avg episode reward: [(0, '273.770'), (1, '279.560')] [2023-10-12 23:22:01,705][44959] Updated weights for policy 1, policy_version 79550 (0.0008) [2023-10-12 23:22:03,966][44958] Updated weights for policy 0, policy_version 79170 (0.0009) [2023-10-12 23:22:04,333][44958] Updated weights for policy 0, policy_version 79180 (0.0009) [2023-10-12 23:22:04,707][44958] Updated weights for policy 0, policy_version 79190 (0.0009) [2023-10-12 23:22:05,090][44958] Updated weights for policy 0, policy_version 79200 (0.0009) [2023-10-12 23:22:05,893][44959] Updated weights for policy 1, policy_version 79560 (0.0010) [2023-10-12 23:22:06,259][44959] Updated weights for policy 1, policy_version 79570 (0.0008) [2023-10-12 23:22:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162562048. Throughput: 0: 1639.3, 1: 1641.2. Samples: 40647800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:06,443][43579] Avg episode reward: [(0, '273.580'), (1, '280.830')] [2023-10-12 23:22:06,629][44959] Updated weights for policy 1, policy_version 79580 (0.0010) [2023-10-12 23:22:09,255][44958] Updated weights for policy 0, policy_version 79210 (0.0008) [2023-10-12 23:22:09,635][44958] Updated weights for policy 0, policy_version 79220 (0.0009) [2023-10-12 23:22:10,006][44958] Updated weights for policy 0, policy_version 79230 (0.0009) [2023-10-12 23:22:10,795][44959] Updated weights for policy 1, policy_version 79590 (0.0009) [2023-10-12 23:22:11,159][44959] Updated weights for policy 1, policy_version 79600 (0.0009) [2023-10-12 23:22:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162627584. Throughput: 0: 1632.0, 1: 1638.4. Samples: 40667106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:11,444][43579] Avg episode reward: [(0, '277.100'), (1, '279.010')] [2023-10-12 23:22:11,531][44959] Updated weights for policy 1, policy_version 79610 (0.0009) [2023-10-12 23:22:14,149][44958] Updated weights for policy 0, policy_version 79240 (0.0008) [2023-10-12 23:22:14,526][44958] Updated weights for policy 0, policy_version 79250 (0.0008) [2023-10-12 23:22:14,905][44958] Updated weights for policy 0, policy_version 79260 (0.0009) [2023-10-12 23:22:15,961][44959] Updated weights for policy 1, policy_version 79620 (0.0008) [2023-10-12 23:22:16,338][44959] Updated weights for policy 1, policy_version 79630 (0.0007) [2023-10-12 23:22:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162693120. Throughput: 0: 1638.9, 1: 1634.9. Samples: 40686896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:16,443][43579] Avg episode reward: [(0, '274.910'), (1, '276.530')] [2023-10-12 23:22:16,696][44959] Updated weights for policy 1, policy_version 79640 (0.0008) [2023-10-12 23:22:19,343][44958] Updated weights for policy 0, policy_version 79270 (0.0009) [2023-10-12 23:22:19,724][44958] Updated weights for policy 0, policy_version 79280 (0.0008) [2023-10-12 23:22:20,094][44958] Updated weights for policy 0, policy_version 79290 (0.0008) [2023-10-12 23:22:20,749][44959] Updated weights for policy 1, policy_version 79650 (0.0008) [2023-10-12 23:22:21,107][44959] Updated weights for policy 1, policy_version 79660 (0.0007) [2023-10-12 23:22:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162758656. Throughput: 0: 1638.5, 1: 1637.4. Samples: 40697020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:21,443][43579] Avg episode reward: [(0, '275.080'), (1, '277.560')] [2023-10-12 23:22:21,483][44959] Updated weights for policy 1, policy_version 79670 (0.0007) [2023-10-12 23:22:21,845][44959] Updated weights for policy 1, policy_version 79680 (0.0010) [2023-10-12 23:22:24,278][44958] Updated weights for policy 0, policy_version 79300 (0.0010) [2023-10-12 23:22:24,646][44958] Updated weights for policy 0, policy_version 79310 (0.0008) [2023-10-12 23:22:25,027][44958] Updated weights for policy 0, policy_version 79320 (0.0008) [2023-10-12 23:22:25,986][44959] Updated weights for policy 1, policy_version 79690 (0.0008) [2023-10-12 23:22:26,362][44959] Updated weights for policy 1, policy_version 79700 (0.0008) [2023-10-12 23:22:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162824192. Throughput: 0: 1638.4, 1: 1645.3. Samples: 40716586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:26,443][43579] Avg episode reward: [(0, '275.420'), (1, '283.340')] [2023-10-12 23:22:26,726][44959] Updated weights for policy 1, policy_version 79710 (0.0010) [2023-10-12 23:22:29,327][44958] Updated weights for policy 0, policy_version 79330 (0.0009) [2023-10-12 23:22:29,725][44958] Updated weights for policy 0, policy_version 79340 (0.0007) [2023-10-12 23:22:30,092][44958] Updated weights for policy 0, policy_version 79350 (0.0009) [2023-10-12 23:22:30,472][44958] Updated weights for policy 0, policy_version 79360 (0.0008) [2023-10-12 23:22:30,967][44959] Updated weights for policy 1, policy_version 79720 (0.0009) [2023-10-12 23:22:31,331][44959] Updated weights for policy 1, policy_version 79730 (0.0009) [2023-10-12 23:22:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162889728. Throughput: 0: 1633.1, 1: 1638.1. Samples: 40735678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:31,443][43579] Avg episode reward: [(0, '278.180'), (1, '283.110')] [2023-10-12 23:22:31,706][44959] Updated weights for policy 1, policy_version 79740 (0.0007) [2023-10-12 23:22:34,653][44958] Updated weights for policy 0, policy_version 79370 (0.0007) [2023-10-12 23:22:35,019][44958] Updated weights for policy 0, policy_version 79380 (0.0007) [2023-10-12 23:22:35,403][44958] Updated weights for policy 0, policy_version 79390 (0.0010) [2023-10-12 23:22:35,690][44959] Updated weights for policy 1, policy_version 79750 (0.0009) [2023-10-12 23:22:36,054][44959] Updated weights for policy 1, policy_version 79760 (0.0009) [2023-10-12 23:22:36,428][44959] Updated weights for policy 1, policy_version 79770 (0.0010) [2023-10-12 23:22:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 162955264. Throughput: 0: 1641.2, 1: 1641.4. Samples: 40746184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:36,443][43579] Avg episode reward: [(0, '275.140'), (1, '286.500')] [2023-10-12 23:22:39,426][44958] Updated weights for policy 0, policy_version 79400 (0.0009) [2023-10-12 23:22:39,800][44958] Updated weights for policy 0, policy_version 79410 (0.0010) [2023-10-12 23:22:40,185][44958] Updated weights for policy 0, policy_version 79420 (0.0009) [2023-10-12 23:22:40,754][44959] Updated weights for policy 1, policy_version 79780 (0.0009) [2023-10-12 23:22:41,124][44959] Updated weights for policy 1, policy_version 79790 (0.0007) [2023-10-12 23:22:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163020800. Throughput: 0: 1643.2, 1: 1642.6. Samples: 40765616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:41,444][43579] Avg episode reward: [(0, '269.790'), (1, '289.160')] [2023-10-12 23:22:41,498][44959] Updated weights for policy 1, policy_version 79800 (0.0010) [2023-10-12 23:22:44,310][44958] Updated weights for policy 0, policy_version 79430 (0.0007) [2023-10-12 23:22:44,686][44958] Updated weights for policy 0, policy_version 79440 (0.0010) [2023-10-12 23:22:45,053][44958] Updated weights for policy 0, policy_version 79450 (0.0007) [2023-10-12 23:22:45,806][44959] Updated weights for policy 1, policy_version 79810 (0.0010) [2023-10-12 23:22:46,166][44959] Updated weights for policy 1, policy_version 79820 (0.0010) [2023-10-12 23:22:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163086336. Throughput: 0: 1634.6, 1: 1641.5. Samples: 40785176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:46,443][43579] Avg episode reward: [(0, '270.030'), (1, '289.100')] [2023-10-12 23:22:46,539][44959] Updated weights for policy 1, policy_version 79830 (0.0010) [2023-10-12 23:22:46,912][44959] Updated weights for policy 1, policy_version 79840 (0.0007) [2023-10-12 23:22:49,123][44958] Updated weights for policy 0, policy_version 79460 (0.0009) [2023-10-12 23:22:49,497][44958] Updated weights for policy 0, policy_version 79470 (0.0008) [2023-10-12 23:22:49,880][44958] Updated weights for policy 0, policy_version 79480 (0.0007) [2023-10-12 23:22:50,791][44959] Updated weights for policy 1, policy_version 79850 (0.0011) [2023-10-12 23:22:51,175][44959] Updated weights for policy 1, policy_version 79860 (0.0009) [2023-10-12 23:22:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163151872. Throughput: 0: 1637.8, 1: 1641.4. Samples: 40795364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:22:51,443][43579] Avg episode reward: [(0, '268.460'), (1, '278.610')] [2023-10-12 23:22:51,544][44959] Updated weights for policy 1, policy_version 79870 (0.0010) [2023-10-12 23:22:53,994][44958] Updated weights for policy 0, policy_version 79490 (0.0007) [2023-10-12 23:22:54,361][44958] Updated weights for policy 0, policy_version 79500 (0.0007) [2023-10-12 23:22:54,733][44958] Updated weights for policy 0, policy_version 79510 (0.0009) [2023-10-12 23:22:55,105][44958] Updated weights for policy 0, policy_version 79520 (0.0007) [2023-10-12 23:22:55,693][44959] Updated weights for policy 1, policy_version 79880 (0.0009) [2023-10-12 23:22:56,063][44959] Updated weights for policy 1, policy_version 79890 (0.0007) [2023-10-12 23:22:56,436][44959] Updated weights for policy 1, policy_version 79900 (0.0007) [2023-10-12 23:22:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163217408. Throughput: 0: 1641.2, 1: 1650.5. Samples: 40815232. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:22:56,443][43579] Avg episode reward: [(0, '263.630'), (1, '275.590')] [2023-10-12 23:22:59,294][44958] Updated weights for policy 0, policy_version 79530 (0.0010) [2023-10-12 23:22:59,669][44958] Updated weights for policy 0, policy_version 79540 (0.0009) [2023-10-12 23:23:00,040][44958] Updated weights for policy 0, policy_version 79550 (0.0009) [2023-10-12 23:23:00,535][44959] Updated weights for policy 1, policy_version 79910 (0.0009) [2023-10-12 23:23:00,895][44959] Updated weights for policy 1, policy_version 79920 (0.0007) [2023-10-12 23:23:01,270][44959] Updated weights for policy 1, policy_version 79930 (0.0008) [2023-10-12 23:23:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163282944. Throughput: 0: 1642.4, 1: 1642.1. Samples: 40834698. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:23:01,443][43579] Avg episode reward: [(0, '263.580'), (1, '272.620')] [2023-10-12 23:23:04,202][44958] Updated weights for policy 0, policy_version 79560 (0.0008) [2023-10-12 23:23:04,570][44958] Updated weights for policy 0, policy_version 79570 (0.0007) [2023-10-12 23:23:04,949][44958] Updated weights for policy 0, policy_version 79580 (0.0007) [2023-10-12 23:23:05,437][44959] Updated weights for policy 1, policy_version 79940 (0.0010) [2023-10-12 23:23:05,799][44959] Updated weights for policy 1, policy_version 79950 (0.0009) [2023-10-12 23:23:06,173][44959] Updated weights for policy 1, policy_version 79960 (0.0011) [2023-10-12 23:23:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163348480. Throughput: 0: 1638.6, 1: 1652.7. Samples: 40845130. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:23:06,443][43579] Avg episode reward: [(0, '271.170'), (1, '266.010')] [2023-10-12 23:23:09,154][44958] Updated weights for policy 0, policy_version 79590 (0.0008) [2023-10-12 23:23:09,531][44958] Updated weights for policy 0, policy_version 79600 (0.0007) [2023-10-12 23:23:09,917][44958] Updated weights for policy 0, policy_version 79610 (0.0009) [2023-10-12 23:23:10,461][44959] Updated weights for policy 1, policy_version 79970 (0.0010) [2023-10-12 23:23:10,880][44959] Updated weights for policy 1, policy_version 79980 (0.0009) [2023-10-12 23:23:11,245][44959] Updated weights for policy 1, policy_version 79990 (0.0009) [2023-10-12 23:23:11,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163414016. Throughput: 0: 1637.6, 1: 1646.6. Samples: 40864374. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:23:11,443][43579] Avg episode reward: [(0, '278.690'), (1, '264.370')] [2023-10-12 23:23:11,612][44959] Updated weights for policy 1, policy_version 80000 (0.0009) [2023-10-12 23:23:14,229][44958] Updated weights for policy 0, policy_version 79620 (0.0009) [2023-10-12 23:23:14,631][44958] Updated weights for policy 0, policy_version 79630 (0.0008) [2023-10-12 23:23:15,006][44958] Updated weights for policy 0, policy_version 79640 (0.0009) [2023-10-12 23:23:15,752][44959] Updated weights for policy 1, policy_version 80010 (0.0008) [2023-10-12 23:23:16,110][44959] Updated weights for policy 1, policy_version 80020 (0.0010) [2023-10-12 23:23:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163479552. Throughput: 0: 1645.2, 1: 1644.1. Samples: 40883694. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:23:16,444][43579] Avg episode reward: [(0, '281.420'), (1, '275.680')] [2023-10-12 23:23:16,489][44959] Updated weights for policy 1, policy_version 80030 (0.0010) [2023-10-12 23:23:19,077][44958] Updated weights for policy 0, policy_version 79650 (0.0007) [2023-10-12 23:23:19,449][44958] Updated weights for policy 0, policy_version 79660 (0.0009) [2023-10-12 23:23:19,821][44958] Updated weights for policy 0, policy_version 79670 (0.0011) [2023-10-12 23:23:20,198][44958] Updated weights for policy 0, policy_version 79680 (0.0009) [2023-10-12 23:23:20,480][44959] Updated weights for policy 1, policy_version 80040 (0.0008) [2023-10-12 23:23:20,852][44959] Updated weights for policy 1, policy_version 80050 (0.0008) [2023-10-12 23:23:21,226][44959] Updated weights for policy 1, policy_version 80060 (0.0008) [2023-10-12 23:23:21,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 163577856. Throughput: 0: 1641.6, 1: 1652.2. Samples: 40894404. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:23:21,444][43579] Avg episode reward: [(0, '287.690'), (1, '269.640')] [2023-10-12 23:23:24,280][44958] Updated weights for policy 0, policy_version 79690 (0.0008) [2023-10-12 23:23:24,641][44958] Updated weights for policy 0, policy_version 79700 (0.0007) [2023-10-12 23:23:25,015][44958] Updated weights for policy 0, policy_version 79710 (0.0008) [2023-10-12 23:23:25,505][44959] Updated weights for policy 1, policy_version 80070 (0.0010) [2023-10-12 23:23:25,864][44959] Updated weights for policy 1, policy_version 80080 (0.0008) [2023-10-12 23:23:26,236][44959] Updated weights for policy 1, policy_version 80090 (0.0010) [2023-10-12 23:23:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163610624. Throughput: 0: 1639.5, 1: 1649.7. Samples: 40913632. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:23:26,443][43579] Avg episode reward: [(0, '289.230'), (1, '266.570')] [2023-10-12 23:23:29,204][44958] Updated weights for policy 0, policy_version 79720 (0.0009) [2023-10-12 23:23:29,568][44958] Updated weights for policy 0, policy_version 79730 (0.0008) [2023-10-12 23:23:29,940][44958] Updated weights for policy 0, policy_version 79740 (0.0011) [2023-10-12 23:23:30,414][44959] Updated weights for policy 1, policy_version 80100 (0.0009) [2023-10-12 23:23:30,785][44959] Updated weights for policy 1, policy_version 80110 (0.0007) [2023-10-12 23:23:31,158][44959] Updated weights for policy 1, policy_version 80120 (0.0009) [2023-10-12 23:23:31,442][43579] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 163676160. Throughput: 0: 1651.9, 1: 1642.6. Samples: 40933428. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:23:31,443][43579] Avg episode reward: [(0, '281.620'), (1, '272.860')] [2023-10-12 23:23:33,853][44958] Updated weights for policy 0, policy_version 79750 (0.0009) [2023-10-12 23:23:34,216][44958] Updated weights for policy 0, policy_version 79760 (0.0008) [2023-10-12 23:23:34,594][44958] Updated weights for policy 0, policy_version 79770 (0.0009) [2023-10-12 23:23:35,194][44959] Updated weights for policy 1, policy_version 80130 (0.0009) [2023-10-12 23:23:35,554][44959] Updated weights for policy 1, policy_version 80140 (0.0007) [2023-10-12 23:23:35,924][44959] Updated weights for policy 1, policy_version 80150 (0.0007) [2023-10-12 23:23:36,295][44959] Updated weights for policy 1, policy_version 80160 (0.0011) [2023-10-12 23:23:36,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 163774464. Throughput: 0: 1645.2, 1: 1652.1. Samples: 40943740. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:23:36,443][43579] Avg episode reward: [(0, '277.310'), (1, '270.450')] [2023-10-12 23:23:39,011][44958] Updated weights for policy 0, policy_version 79780 (0.0009) [2023-10-12 23:23:39,375][44958] Updated weights for policy 0, policy_version 79790 (0.0011) [2023-10-12 23:23:39,745][44958] Updated weights for policy 0, policy_version 79800 (0.0010) [2023-10-12 23:23:40,497][44959] Updated weights for policy 1, policy_version 80170 (0.0008) [2023-10-12 23:23:40,859][44959] Updated weights for policy 1, policy_version 80180 (0.0009) [2023-10-12 23:23:41,221][44959] Updated weights for policy 1, policy_version 80190 (0.0008) [2023-10-12 23:23:41,443][43579] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 163840000. Throughput: 0: 1642.7, 1: 1647.9. Samples: 40963310. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-12 23:23:41,444][43579] Avg episode reward: [(0, '270.520'), (1, '265.870')] [2023-10-12 23:23:43,922][44958] Updated weights for policy 0, policy_version 79810 (0.0009) [2023-10-12 23:23:44,302][44958] Updated weights for policy 0, policy_version 79820 (0.0008) [2023-10-12 23:23:44,671][44958] Updated weights for policy 0, policy_version 79830 (0.0009) [2023-10-12 23:23:45,043][44958] Updated weights for policy 0, policy_version 79840 (0.0010) [2023-10-12 23:23:45,156][44959] Updated weights for policy 1, policy_version 80200 (0.0008) [2023-10-12 23:23:45,528][44959] Updated weights for policy 1, policy_version 80210 (0.0008) [2023-10-12 23:23:45,890][44959] Updated weights for policy 1, policy_version 80220 (0.0008) [2023-10-12 23:23:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 163905536. Throughput: 0: 1642.8, 1: 1645.2. Samples: 40982662. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:23:46,443][43579] Avg episode reward: [(0, '269.300'), (1, '276.930')] [2023-10-12 23:23:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000080224_82149376.pth... [2023-10-12 23:23:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000079840_81756160.pth... [2023-10-12 23:23:46,485][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000078688_80576512.pth [2023-10-12 23:23:46,490][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000078304_80183296.pth [2023-10-12 23:23:49,248][44958] Updated weights for policy 0, policy_version 79850 (0.0011) [2023-10-12 23:23:49,619][44958] Updated weights for policy 0, policy_version 79860 (0.0008) [2023-10-12 23:23:49,985][44958] Updated weights for policy 0, policy_version 79870 (0.0007) [2023-10-12 23:23:50,151][44959] Updated weights for policy 1, policy_version 80230 (0.0008) [2023-10-12 23:23:50,512][44959] Updated weights for policy 1, policy_version 80240 (0.0009) [2023-10-12 23:23:50,881][44959] Updated weights for policy 1, policy_version 80250 (0.0008) [2023-10-12 23:23:51,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 163971072. Throughput: 0: 1641.7, 1: 1658.0. Samples: 40993616. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:23:51,443][43579] Avg episode reward: [(0, '267.260'), (1, '279.670')] [2023-10-12 23:23:54,058][44958] Updated weights for policy 0, policy_version 79880 (0.0007) [2023-10-12 23:23:54,439][44958] Updated weights for policy 0, policy_version 79890 (0.0007) [2023-10-12 23:23:54,812][44958] Updated weights for policy 0, policy_version 79900 (0.0008) [2023-10-12 23:23:55,107][44959] Updated weights for policy 1, policy_version 80260 (0.0008) [2023-10-12 23:23:55,517][44959] Updated weights for policy 1, policy_version 80270 (0.0010) [2023-10-12 23:23:55,880][44959] Updated weights for policy 1, policy_version 80280 (0.0007) [2023-10-12 23:23:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 164036608. Throughput: 0: 1645.8, 1: 1658.0. Samples: 41013044. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:23:56,443][43579] Avg episode reward: [(0, '263.240'), (1, '276.420')] [2023-10-12 23:23:59,216][44958] Updated weights for policy 0, policy_version 79910 (0.0009) [2023-10-12 23:23:59,600][44958] Updated weights for policy 0, policy_version 79920 (0.0009) [2023-10-12 23:23:59,982][44958] Updated weights for policy 0, policy_version 79930 (0.0011) [2023-10-12 23:24:00,059][44959] Updated weights for policy 1, policy_version 80290 (0.0007) [2023-10-12 23:24:00,421][44959] Updated weights for policy 1, policy_version 80300 (0.0011) [2023-10-12 23:24:00,787][44959] Updated weights for policy 1, policy_version 80310 (0.0008) [2023-10-12 23:24:01,152][44959] Updated weights for policy 1, policy_version 80320 (0.0009) [2023-10-12 23:24:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 164102144. Throughput: 0: 1643.4, 1: 1650.1. Samples: 41031900. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:24:01,443][43579] Avg episode reward: [(0, '268.670'), (1, '275.350')] [2023-10-12 23:24:04,109][44958] Updated weights for policy 0, policy_version 79940 (0.0010) [2023-10-12 23:24:04,487][44958] Updated weights for policy 0, policy_version 79950 (0.0010) [2023-10-12 23:24:04,867][44958] Updated weights for policy 0, policy_version 79960 (0.0008) [2023-10-12 23:24:05,349][44959] Updated weights for policy 1, policy_version 80330 (0.0007) [2023-10-12 23:24:05,708][44959] Updated weights for policy 1, policy_version 80340 (0.0007) [2023-10-12 23:24:06,087][44959] Updated weights for policy 1, policy_version 80350 (0.0008) [2023-10-12 23:24:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 164167680. Throughput: 0: 1639.8, 1: 1658.7. Samples: 41042834. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:24:06,443][43579] Avg episode reward: [(0, '269.000'), (1, '270.760')] [2023-10-12 23:24:08,995][44958] Updated weights for policy 0, policy_version 79970 (0.0008) [2023-10-12 23:24:09,372][44958] Updated weights for policy 0, policy_version 79980 (0.0009) [2023-10-12 23:24:09,740][44958] Updated weights for policy 0, policy_version 79990 (0.0007) [2023-10-12 23:24:10,090][44959] Updated weights for policy 1, policy_version 80360 (0.0009) [2023-10-12 23:24:10,116][44958] Updated weights for policy 0, policy_version 80000 (0.0008) [2023-10-12 23:24:10,458][44959] Updated weights for policy 1, policy_version 80370 (0.0010) [2023-10-12 23:24:10,823][44959] Updated weights for policy 1, policy_version 80380 (0.0007) [2023-10-12 23:24:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 164233216. Throughput: 0: 1640.8, 1: 1657.2. Samples: 41062046. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:24:11,444][43579] Avg episode reward: [(0, '270.090'), (1, '267.310')] [2023-10-12 23:24:14,446][44958] Updated weights for policy 0, policy_version 80010 (0.0009) [2023-10-12 23:24:14,806][44958] Updated weights for policy 0, policy_version 80020 (0.0009) [2023-10-12 23:24:15,065][44959] Updated weights for policy 1, policy_version 80390 (0.0008) [2023-10-12 23:24:15,173][44958] Updated weights for policy 0, policy_version 80030 (0.0008) [2023-10-12 23:24:15,427][44959] Updated weights for policy 1, policy_version 80400 (0.0009) [2023-10-12 23:24:15,794][44959] Updated weights for policy 1, policy_version 80410 (0.0010) [2023-10-12 23:24:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 164298752. Throughput: 0: 1627.4, 1: 1650.7. Samples: 41080940. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:24:16,444][43579] Avg episode reward: [(0, '277.930'), (1, '268.590')] [2023-10-12 23:24:19,480][44958] Updated weights for policy 0, policy_version 80040 (0.0008) [2023-10-12 23:24:19,846][44958] Updated weights for policy 0, policy_version 80050 (0.0008) [2023-10-12 23:24:19,887][44959] Updated weights for policy 1, policy_version 80420 (0.0009) [2023-10-12 23:24:20,219][44958] Updated weights for policy 0, policy_version 80060 (0.0009) [2023-10-12 23:24:20,254][44959] Updated weights for policy 1, policy_version 80430 (0.0009) [2023-10-12 23:24:20,623][44959] Updated weights for policy 1, policy_version 80440 (0.0009) [2023-10-12 23:24:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164364288. Throughput: 0: 1635.9, 1: 1661.0. Samples: 41092098. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:24:21,444][43579] Avg episode reward: [(0, '278.920'), (1, '268.750')] [2023-10-12 23:24:24,411][44958] Updated weights for policy 0, policy_version 80070 (0.0007) [2023-10-12 23:24:24,782][44958] Updated weights for policy 0, policy_version 80080 (0.0008) [2023-10-12 23:24:24,787][44959] Updated weights for policy 1, policy_version 80450 (0.0009) [2023-10-12 23:24:25,158][44959] Updated weights for policy 1, policy_version 80460 (0.0007) [2023-10-12 23:24:25,160][44958] Updated weights for policy 0, policy_version 80090 (0.0008) [2023-10-12 23:24:25,519][44959] Updated weights for policy 1, policy_version 80470 (0.0008) [2023-10-12 23:24:25,887][44959] Updated weights for policy 1, policy_version 80480 (0.0011) [2023-10-12 23:24:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 164429824. Throughput: 0: 1632.8, 1: 1646.9. Samples: 41110894. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:24:26,443][43579] Avg episode reward: [(0, '284.230'), (1, '277.490')] [2023-10-12 23:24:29,205][44958] Updated weights for policy 0, policy_version 80100 (0.0008) [2023-10-12 23:24:29,575][44958] Updated weights for policy 0, policy_version 80110 (0.0007) [2023-10-12 23:24:29,952][44958] Updated weights for policy 0, policy_version 80120 (0.0009) [2023-10-12 23:24:30,208][44959] Updated weights for policy 1, policy_version 80490 (0.0007) [2023-10-12 23:24:30,580][44959] Updated weights for policy 1, policy_version 80500 (0.0010) [2023-10-12 23:24:30,944][44959] Updated weights for policy 1, policy_version 80510 (0.0010) [2023-10-12 23:24:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 164495360. Throughput: 0: 1625.4, 1: 1647.5. Samples: 41129942. Policy #0 lag: (min: 17.0, avg: 33.6, max: 49.0) [2023-10-12 23:24:31,444][43579] Avg episode reward: [(0, '278.450'), (1, '284.030')] [2023-10-12 23:24:34,144][44958] Updated weights for policy 0, policy_version 80130 (0.0008) [2023-10-12 23:24:34,518][44958] Updated weights for policy 0, policy_version 80140 (0.0009) [2023-10-12 23:24:34,887][44958] Updated weights for policy 0, policy_version 80150 (0.0009) [2023-10-12 23:24:34,963][44959] Updated weights for policy 1, policy_version 80520 (0.0010) [2023-10-12 23:24:35,249][44958] Updated weights for policy 0, policy_version 80160 (0.0009) [2023-10-12 23:24:35,329][44959] Updated weights for policy 1, policy_version 80530 (0.0010) [2023-10-12 23:24:35,694][44959] Updated weights for policy 1, policy_version 80540 (0.0010) [2023-10-12 23:24:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164560896. Throughput: 0: 1627.1, 1: 1644.5. Samples: 41140840. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:24:36,443][43579] Avg episode reward: [(0, '278.980'), (1, '286.000')] [2023-10-12 23:24:39,398][44958] Updated weights for policy 0, policy_version 80170 (0.0009) [2023-10-12 23:24:39,763][44958] Updated weights for policy 0, policy_version 80180 (0.0008) [2023-10-12 23:24:40,111][44959] Updated weights for policy 1, policy_version 80550 (0.0009) [2023-10-12 23:24:40,136][44958] Updated weights for policy 0, policy_version 80190 (0.0009) [2023-10-12 23:24:40,482][44959] Updated weights for policy 1, policy_version 80560 (0.0007) [2023-10-12 23:24:40,847][44959] Updated weights for policy 1, policy_version 80570 (0.0008) [2023-10-12 23:24:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164626432. Throughput: 0: 1627.1, 1: 1637.3. Samples: 41159942. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:24:41,443][43579] Avg episode reward: [(0, '272.330'), (1, '290.880')] [2023-10-12 23:24:44,492][44958] Updated weights for policy 0, policy_version 80200 (0.0010) [2023-10-12 23:24:44,872][44958] Updated weights for policy 0, policy_version 80210 (0.0009) [2023-10-12 23:24:44,998][44959] Updated weights for policy 1, policy_version 80580 (0.0008) [2023-10-12 23:24:45,240][44958] Updated weights for policy 0, policy_version 80220 (0.0009) [2023-10-12 23:24:45,396][44959] Updated weights for policy 1, policy_version 80590 (0.0007) [2023-10-12 23:24:45,761][44959] Updated weights for policy 1, policy_version 80600 (0.0010) [2023-10-12 23:24:46,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164691968. Throughput: 0: 1626.9, 1: 1639.0. Samples: 41178864. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:24:46,443][43579] Avg episode reward: [(0, '268.130'), (1, '289.120')] [2023-10-12 23:24:49,341][44958] Updated weights for policy 0, policy_version 80230 (0.0010) [2023-10-12 23:24:49,703][44958] Updated weights for policy 0, policy_version 80240 (0.0011) [2023-10-12 23:24:49,870][44959] Updated weights for policy 1, policy_version 80610 (0.0008) [2023-10-12 23:24:50,076][44958] Updated weights for policy 0, policy_version 80250 (0.0008) [2023-10-12 23:24:50,227][44959] Updated weights for policy 1, policy_version 80620 (0.0009) [2023-10-12 23:24:50,599][44959] Updated weights for policy 1, policy_version 80630 (0.0009) [2023-10-12 23:24:50,960][44959] Updated weights for policy 1, policy_version 80640 (0.0009) [2023-10-12 23:24:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164757504. Throughput: 0: 1636.8, 1: 1635.1. Samples: 41190066. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:24:51,443][43579] Avg episode reward: [(0, '266.730'), (1, '289.030')] [2023-10-12 23:24:54,182][44958] Updated weights for policy 0, policy_version 80260 (0.0008) [2023-10-12 23:24:54,548][44958] Updated weights for policy 0, policy_version 80270 (0.0007) [2023-10-12 23:24:54,921][44958] Updated weights for policy 0, policy_version 80280 (0.0007) [2023-10-12 23:24:55,145][44959] Updated weights for policy 1, policy_version 80650 (0.0009) [2023-10-12 23:24:55,518][44959] Updated weights for policy 1, policy_version 80660 (0.0008) [2023-10-12 23:24:55,885][44959] Updated weights for policy 1, policy_version 80670 (0.0008) [2023-10-12 23:24:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164823040. Throughput: 0: 1636.3, 1: 1634.0. Samples: 41209206. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:24:56,443][43579] Avg episode reward: [(0, '268.010'), (1, '289.630')] [2023-10-12 23:24:59,108][44958] Updated weights for policy 0, policy_version 80290 (0.0009) [2023-10-12 23:24:59,481][44958] Updated weights for policy 0, policy_version 80300 (0.0007) [2023-10-12 23:24:59,848][44958] Updated weights for policy 0, policy_version 80310 (0.0008) [2023-10-12 23:24:59,947][44959] Updated weights for policy 1, policy_version 80680 (0.0007) [2023-10-12 23:25:00,219][44958] Updated weights for policy 0, policy_version 80320 (0.0008) [2023-10-12 23:25:00,318][44959] Updated weights for policy 1, policy_version 80690 (0.0007) [2023-10-12 23:25:00,678][44959] Updated weights for policy 1, policy_version 80700 (0.0010) [2023-10-12 23:25:01,442][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164888576. Throughput: 0: 1637.9, 1: 1638.2. Samples: 41228366. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:25:01,443][43579] Avg episode reward: [(0, '269.150'), (1, '289.510')] [2023-10-12 23:25:04,403][44958] Updated weights for policy 0, policy_version 80330 (0.0007) [2023-10-12 23:25:04,770][44958] Updated weights for policy 0, policy_version 80340 (0.0007) [2023-10-12 23:25:04,790][44959] Updated weights for policy 1, policy_version 80710 (0.0010) [2023-10-12 23:25:05,145][44958] Updated weights for policy 0, policy_version 80350 (0.0008) [2023-10-12 23:25:05,154][44959] Updated weights for policy 1, policy_version 80720 (0.0009) [2023-10-12 23:25:05,513][44959] Updated weights for policy 1, policy_version 80730 (0.0010) [2023-10-12 23:25:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 164954112. Throughput: 0: 1638.0, 1: 1639.1. Samples: 41239564. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:25:06,444][43579] Avg episode reward: [(0, '267.690'), (1, '287.860')] [2023-10-12 23:25:09,221][44958] Updated weights for policy 0, policy_version 80360 (0.0009) [2023-10-12 23:25:09,599][44958] Updated weights for policy 0, policy_version 80370 (0.0008) [2023-10-12 23:25:09,726][44959] Updated weights for policy 1, policy_version 80740 (0.0010) [2023-10-12 23:25:09,979][44958] Updated weights for policy 0, policy_version 80380 (0.0010) [2023-10-12 23:25:10,093][44959] Updated weights for policy 1, policy_version 80750 (0.0007) [2023-10-12 23:25:10,456][44959] Updated weights for policy 1, policy_version 80760 (0.0010) [2023-10-12 23:25:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165019648. Throughput: 0: 1636.2, 1: 1640.5. Samples: 41258346. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:25:11,443][43579] Avg episode reward: [(0, '273.020'), (1, '286.790')] [2023-10-12 23:25:14,225][44958] Updated weights for policy 0, policy_version 80390 (0.0009) [2023-10-12 23:25:14,547][44959] Updated weights for policy 1, policy_version 80770 (0.0008) [2023-10-12 23:25:14,596][44958] Updated weights for policy 0, policy_version 80400 (0.0009) [2023-10-12 23:25:14,925][44959] Updated weights for policy 1, policy_version 80780 (0.0007) [2023-10-12 23:25:14,975][44958] Updated weights for policy 0, policy_version 80410 (0.0008) [2023-10-12 23:25:15,290][44959] Updated weights for policy 1, policy_version 80790 (0.0009) [2023-10-12 23:25:15,656][44959] Updated weights for policy 1, policy_version 80800 (0.0008) [2023-10-12 23:25:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165085184. Throughput: 0: 1639.0, 1: 1644.4. Samples: 41277696. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:25:16,443][43579] Avg episode reward: [(0, '274.070'), (1, '285.730')] [2023-10-12 23:25:19,313][44958] Updated weights for policy 0, policy_version 80420 (0.0009) [2023-10-12 23:25:19,686][44958] Updated weights for policy 0, policy_version 80430 (0.0008) [2023-10-12 23:25:19,847][44959] Updated weights for policy 1, policy_version 80810 (0.0009) [2023-10-12 23:25:20,057][44958] Updated weights for policy 0, policy_version 80440 (0.0008) [2023-10-12 23:25:20,211][44959] Updated weights for policy 1, policy_version 80820 (0.0009) [2023-10-12 23:25:20,575][44959] Updated weights for policy 1, policy_version 80830 (0.0008) [2023-10-12 23:25:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165150720. Throughput: 0: 1647.5, 1: 1646.2. Samples: 41289056. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-12 23:25:21,443][43579] Avg episode reward: [(0, '274.800'), (1, '282.630')] [2023-10-12 23:25:24,376][44958] Updated weights for policy 0, policy_version 80450 (0.0008) [2023-10-12 23:25:24,741][44958] Updated weights for policy 0, policy_version 80460 (0.0007) [2023-10-12 23:25:24,909][44959] Updated weights for policy 1, policy_version 80840 (0.0009) [2023-10-12 23:25:25,111][44958] Updated weights for policy 0, policy_version 80470 (0.0008) [2023-10-12 23:25:25,276][44959] Updated weights for policy 1, policy_version 80850 (0.0009) [2023-10-12 23:25:25,483][44958] Updated weights for policy 0, policy_version 80480 (0.0008) [2023-10-12 23:25:25,646][44959] Updated weights for policy 1, policy_version 80860 (0.0008) [2023-10-12 23:25:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165216256. Throughput: 0: 1644.8, 1: 1640.2. Samples: 41307768. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:25:26,444][43579] Avg episode reward: [(0, '274.260'), (1, '278.690')] [2023-10-12 23:25:29,790][44958] Updated weights for policy 0, policy_version 80490 (0.0007) [2023-10-12 23:25:29,840][44959] Updated weights for policy 1, policy_version 80870 (0.0009) [2023-10-12 23:25:30,170][44958] Updated weights for policy 0, policy_version 80500 (0.0009) [2023-10-12 23:25:30,222][44959] Updated weights for policy 1, policy_version 80880 (0.0008) [2023-10-12 23:25:30,531][44958] Updated weights for policy 0, policy_version 80510 (0.0010) [2023-10-12 23:25:30,599][44959] Updated weights for policy 1, policy_version 80890 (0.0007) [2023-10-12 23:25:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165281792. Throughput: 0: 1637.3, 1: 1643.5. Samples: 41326498. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:25:31,444][43579] Avg episode reward: [(0, '273.420'), (1, '281.060')] [2023-10-12 23:25:34,304][44958] Updated weights for policy 0, policy_version 80520 (0.0007) [2023-10-12 23:25:34,683][44958] Updated weights for policy 0, policy_version 80530 (0.0007) [2023-10-12 23:25:34,713][44959] Updated weights for policy 1, policy_version 80900 (0.0008) [2023-10-12 23:25:35,054][44958] Updated weights for policy 0, policy_version 80540 (0.0008) [2023-10-12 23:25:35,086][44959] Updated weights for policy 1, policy_version 80910 (0.0009) [2023-10-12 23:25:35,452][44959] Updated weights for policy 1, policy_version 80920 (0.0007) [2023-10-12 23:25:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165347328. Throughput: 0: 1633.6, 1: 1648.6. Samples: 41337764. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:25:36,443][43579] Avg episode reward: [(0, '279.170'), (1, '278.310')] [2023-10-12 23:25:39,324][44958] Updated weights for policy 0, policy_version 80550 (0.0007) [2023-10-12 23:25:39,630][44959] Updated weights for policy 1, policy_version 80930 (0.0010) [2023-10-12 23:25:39,695][44958] Updated weights for policy 0, policy_version 80560 (0.0008) [2023-10-12 23:25:39,992][44959] Updated weights for policy 1, policy_version 80940 (0.0007) [2023-10-12 23:25:40,071][44958] Updated weights for policy 0, policy_version 80570 (0.0009) [2023-10-12 23:25:40,372][44959] Updated weights for policy 1, policy_version 80950 (0.0009) [2023-10-12 23:25:40,730][44959] Updated weights for policy 1, policy_version 80960 (0.0010) [2023-10-12 23:25:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165412864. Throughput: 0: 1632.8, 1: 1642.4. Samples: 41356592. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:25:41,443][43579] Avg episode reward: [(0, '282.230'), (1, '280.040')] [2023-10-12 23:25:44,221][44958] Updated weights for policy 0, policy_version 80580 (0.0008) [2023-10-12 23:25:44,584][44958] Updated weights for policy 0, policy_version 80590 (0.0007) [2023-10-12 23:25:44,957][44958] Updated weights for policy 0, policy_version 80600 (0.0007) [2023-10-12 23:25:45,023][44959] Updated weights for policy 1, policy_version 80970 (0.0007) [2023-10-12 23:25:45,385][44959] Updated weights for policy 1, policy_version 80980 (0.0008) [2023-10-12 23:25:45,755][44959] Updated weights for policy 1, policy_version 80990 (0.0009) [2023-10-12 23:25:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165478400. Throughput: 0: 1634.6, 1: 1639.1. Samples: 41375682. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:25:46,444][43579] Avg episode reward: [(0, '281.550'), (1, '278.260')] [2023-10-12 23:25:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000080992_82935808.pth... [2023-10-12 23:25:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000080608_82542592.pth... [2023-10-12 23:25:46,483][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000079456_81362944.pth [2023-10-12 23:25:46,494][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000079072_80969728.pth [2023-10-12 23:25:49,127][44958] Updated weights for policy 0, policy_version 80610 (0.0007) [2023-10-12 23:25:49,501][44958] Updated weights for policy 0, policy_version 80620 (0.0009) [2023-10-12 23:25:49,872][44958] Updated weights for policy 0, policy_version 80630 (0.0008) [2023-10-12 23:25:49,978][44959] Updated weights for policy 1, policy_version 81000 (0.0008) [2023-10-12 23:25:50,242][44958] Updated weights for policy 0, policy_version 80640 (0.0009) [2023-10-12 23:25:50,349][44959] Updated weights for policy 1, policy_version 81010 (0.0007) [2023-10-12 23:25:50,717][44959] Updated weights for policy 1, policy_version 81020 (0.0007) [2023-10-12 23:25:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165543936. Throughput: 0: 1636.5, 1: 1636.2. Samples: 41386836. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:25:51,444][43579] Avg episode reward: [(0, '278.010'), (1, '279.320')] [2023-10-12 23:25:54,545][44958] Updated weights for policy 0, policy_version 80650 (0.0009) [2023-10-12 23:25:54,838][44959] Updated weights for policy 1, policy_version 81030 (0.0008) [2023-10-12 23:25:54,919][44958] Updated weights for policy 0, policy_version 80660 (0.0008) [2023-10-12 23:25:55,203][44959] Updated weights for policy 1, policy_version 81040 (0.0008) [2023-10-12 23:25:55,286][44958] Updated weights for policy 0, policy_version 80670 (0.0009) [2023-10-12 23:25:55,572][44959] Updated weights for policy 1, policy_version 81050 (0.0007) [2023-10-12 23:25:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165609472. Throughput: 0: 1636.8, 1: 1635.8. Samples: 41405614. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:25:56,444][43579] Avg episode reward: [(0, '277.220'), (1, '281.470')] [2023-10-12 23:25:59,521][44958] Updated weights for policy 0, policy_version 80680 (0.0008) [2023-10-12 23:25:59,582][44959] Updated weights for policy 1, policy_version 81060 (0.0007) [2023-10-12 23:25:59,886][44958] Updated weights for policy 0, policy_version 80690 (0.0007) [2023-10-12 23:25:59,948][44959] Updated weights for policy 1, policy_version 81070 (0.0007) [2023-10-12 23:26:00,266][44958] Updated weights for policy 0, policy_version 80700 (0.0010) [2023-10-12 23:26:00,325][44959] Updated weights for policy 1, policy_version 81080 (0.0008) [2023-10-12 23:26:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165675008. Throughput: 0: 1630.3, 1: 1636.0. Samples: 41424676. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:26:01,443][43579] Avg episode reward: [(0, '277.200'), (1, '280.580')] [2023-10-12 23:26:04,355][44958] Updated weights for policy 0, policy_version 80710 (0.0009) [2023-10-12 23:26:04,428][44959] Updated weights for policy 1, policy_version 81090 (0.0008) [2023-10-12 23:26:04,724][44958] Updated weights for policy 0, policy_version 80720 (0.0010) [2023-10-12 23:26:04,794][44959] Updated weights for policy 1, policy_version 81100 (0.0007) [2023-10-12 23:26:05,098][44958] Updated weights for policy 0, policy_version 80730 (0.0009) [2023-10-12 23:26:05,166][44959] Updated weights for policy 1, policy_version 81110 (0.0009) [2023-10-12 23:26:05,542][44959] Updated weights for policy 1, policy_version 81120 (0.0010) [2023-10-12 23:26:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165740544. Throughput: 0: 1631.4, 1: 1635.9. Samples: 41436086. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:26:06,444][43579] Avg episode reward: [(0, '274.590'), (1, '278.810')] [2023-10-12 23:26:09,304][44958] Updated weights for policy 0, policy_version 80740 (0.0009) [2023-10-12 23:26:09,675][44958] Updated weights for policy 0, policy_version 80750 (0.0009) [2023-10-12 23:26:09,809][44959] Updated weights for policy 1, policy_version 81130 (0.0007) [2023-10-12 23:26:10,046][44958] Updated weights for policy 0, policy_version 80760 (0.0009) [2023-10-12 23:26:10,179][44959] Updated weights for policy 1, policy_version 81140 (0.0007) [2023-10-12 23:26:10,544][44959] Updated weights for policy 1, policy_version 81150 (0.0008) [2023-10-12 23:26:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165806080. Throughput: 0: 1626.8, 1: 1632.4. Samples: 41454430. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-12 23:26:11,443][43579] Avg episode reward: [(0, '266.540'), (1, '278.220')] [2023-10-12 23:26:14,321][44958] Updated weights for policy 0, policy_version 80770 (0.0009) [2023-10-12 23:26:14,732][44958] Updated weights for policy 0, policy_version 80780 (0.0011) [2023-10-12 23:26:14,995][44959] Updated weights for policy 1, policy_version 81160 (0.0010) [2023-10-12 23:26:15,107][44958] Updated weights for policy 0, policy_version 80790 (0.0008) [2023-10-12 23:26:15,359][44959] Updated weights for policy 1, policy_version 81170 (0.0007) [2023-10-12 23:26:15,480][44958] Updated weights for policy 0, policy_version 80800 (0.0009) [2023-10-12 23:26:15,724][44959] Updated weights for policy 1, policy_version 81180 (0.0009) [2023-10-12 23:26:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165871616. Throughput: 0: 1631.9, 1: 1630.4. Samples: 41473300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:26:16,443][43579] Avg episode reward: [(0, '265.590'), (1, '276.060')] [2023-10-12 23:26:19,528][44958] Updated weights for policy 0, policy_version 80810 (0.0009) [2023-10-12 23:26:19,900][44958] Updated weights for policy 0, policy_version 80820 (0.0008) [2023-10-12 23:26:19,921][44959] Updated weights for policy 1, policy_version 81190 (0.0010) [2023-10-12 23:26:20,269][44958] Updated weights for policy 0, policy_version 80830 (0.0008) [2023-10-12 23:26:20,280][44959] Updated weights for policy 1, policy_version 81200 (0.0008) [2023-10-12 23:26:20,655][44959] Updated weights for policy 1, policy_version 81210 (0.0008) [2023-10-12 23:26:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 165937152. Throughput: 0: 1635.1, 1: 1630.8. Samples: 41484730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:26:21,443][43579] Avg episode reward: [(0, '271.390'), (1, '275.150')] [2023-10-12 23:26:24,570][44958] Updated weights for policy 0, policy_version 80840 (0.0007) [2023-10-12 23:26:24,949][44958] Updated weights for policy 0, policy_version 80850 (0.0008) [2023-10-12 23:26:24,954][44959] Updated weights for policy 1, policy_version 81220 (0.0008) [2023-10-12 23:26:25,322][44958] Updated weights for policy 0, policy_version 80860 (0.0007) [2023-10-12 23:26:25,327][44959] Updated weights for policy 1, policy_version 81230 (0.0008) [2023-10-12 23:26:25,692][44959] Updated weights for policy 1, policy_version 81240 (0.0010) [2023-10-12 23:26:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166002688. Throughput: 0: 1635.1, 1: 1632.8. Samples: 41503650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:26:26,444][43579] Avg episode reward: [(0, '271.210'), (1, '276.430')] [2023-10-12 23:26:29,430][44958] Updated weights for policy 0, policy_version 80870 (0.0008) [2023-10-12 23:26:29,788][44959] Updated weights for policy 1, policy_version 81250 (0.0008) [2023-10-12 23:26:29,797][44958] Updated weights for policy 0, policy_version 80880 (0.0009) [2023-10-12 23:26:30,150][44959] Updated weights for policy 1, policy_version 81260 (0.0008) [2023-10-12 23:26:30,165][44958] Updated weights for policy 0, policy_version 80890 (0.0009) [2023-10-12 23:26:30,523][44959] Updated weights for policy 1, policy_version 81270 (0.0008) [2023-10-12 23:26:30,886][44959] Updated weights for policy 1, policy_version 81280 (0.0007) [2023-10-12 23:26:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166068224. Throughput: 0: 1628.8, 1: 1635.2. Samples: 41522560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:26:31,443][43579] Avg episode reward: [(0, '274.410'), (1, '276.670')] [2023-10-12 23:26:34,524][44958] Updated weights for policy 0, policy_version 80900 (0.0009) [2023-10-12 23:26:34,887][44959] Updated weights for policy 1, policy_version 81290 (0.0008) [2023-10-12 23:26:34,894][44958] Updated weights for policy 0, policy_version 80910 (0.0010) [2023-10-12 23:26:35,254][44959] Updated weights for policy 1, policy_version 81300 (0.0007) [2023-10-12 23:26:35,259][44958] Updated weights for policy 0, policy_version 80920 (0.0009) [2023-10-12 23:26:35,616][44959] Updated weights for policy 1, policy_version 81310 (0.0007) [2023-10-12 23:26:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166133760. Throughput: 0: 1628.9, 1: 1644.3. Samples: 41534130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:26:36,444][43579] Avg episode reward: [(0, '276.540'), (1, '280.160')] [2023-10-12 23:26:39,415][44958] Updated weights for policy 0, policy_version 80930 (0.0009) [2023-10-12 23:26:39,724][44959] Updated weights for policy 1, policy_version 81320 (0.0009) [2023-10-12 23:26:39,785][44958] Updated weights for policy 0, policy_version 80940 (0.0007) [2023-10-12 23:26:40,086][44959] Updated weights for policy 1, policy_version 81330 (0.0009) [2023-10-12 23:26:40,163][44958] Updated weights for policy 0, policy_version 80950 (0.0007) [2023-10-12 23:26:40,461][44959] Updated weights for policy 1, policy_version 81340 (0.0009) [2023-10-12 23:26:40,539][44958] Updated weights for policy 0, policy_version 80960 (0.0008) [2023-10-12 23:26:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166199296. Throughput: 0: 1633.3, 1: 1634.9. Samples: 41552682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:26:41,443][43579] Avg episode reward: [(0, '274.210'), (1, '281.030')] [2023-10-12 23:26:44,578][44959] Updated weights for policy 1, policy_version 81350 (0.0008) [2023-10-12 23:26:44,861][44958] Updated weights for policy 0, policy_version 80970 (0.0007) [2023-10-12 23:26:44,939][44959] Updated weights for policy 1, policy_version 81360 (0.0009) [2023-10-12 23:26:45,233][44958] Updated weights for policy 0, policy_version 80980 (0.0008) [2023-10-12 23:26:45,301][44959] Updated weights for policy 1, policy_version 81370 (0.0008) [2023-10-12 23:26:45,591][44958] Updated weights for policy 0, policy_version 80990 (0.0008) [2023-10-12 23:26:46,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166264832. Throughput: 0: 1628.7, 1: 1641.4. Samples: 41571830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:26:46,443][43579] Avg episode reward: [(0, '271.080'), (1, '279.780')] [2023-10-12 23:26:49,550][44959] Updated weights for policy 1, policy_version 81380 (0.0009) [2023-10-12 23:26:49,913][44959] Updated weights for policy 1, policy_version 81390 (0.0007) [2023-10-12 23:26:49,961][44958] Updated weights for policy 0, policy_version 81000 (0.0008) [2023-10-12 23:26:50,274][44959] Updated weights for policy 1, policy_version 81400 (0.0009) [2023-10-12 23:26:50,326][44958] Updated weights for policy 0, policy_version 81010 (0.0009) [2023-10-12 23:26:50,692][44958] Updated weights for policy 0, policy_version 81020 (0.0008) [2023-10-12 23:26:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166330368. Throughput: 0: 1626.1, 1: 1642.6. Samples: 41583178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:26:51,443][43579] Avg episode reward: [(0, '274.340'), (1, '281.110')] [2023-10-12 23:26:54,479][44959] Updated weights for policy 1, policy_version 81410 (0.0009) [2023-10-12 23:26:54,692][44958] Updated weights for policy 0, policy_version 81030 (0.0008) [2023-10-12 23:26:54,843][44959] Updated weights for policy 1, policy_version 81420 (0.0008) [2023-10-12 23:26:55,056][44958] Updated weights for policy 0, policy_version 81040 (0.0007) [2023-10-12 23:26:55,216][44959] Updated weights for policy 1, policy_version 81430 (0.0008) [2023-10-12 23:26:55,430][44958] Updated weights for policy 0, policy_version 81050 (0.0009) [2023-10-12 23:26:55,587][44959] Updated weights for policy 1, policy_version 81440 (0.0008) [2023-10-12 23:26:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 166395904. Throughput: 0: 1634.6, 1: 1647.3. Samples: 41602116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:26:56,443][43579] Avg episode reward: [(0, '271.930'), (1, '280.570')] [2023-10-12 23:26:59,762][44958] Updated weights for policy 0, policy_version 81060 (0.0008) [2023-10-12 23:26:59,863][44959] Updated weights for policy 1, policy_version 81450 (0.0010) [2023-10-12 23:27:00,155][44958] Updated weights for policy 0, policy_version 81070 (0.0009) [2023-10-12 23:27:00,251][44959] Updated weights for policy 1, policy_version 81460 (0.0009) [2023-10-12 23:27:00,527][44958] Updated weights for policy 0, policy_version 81080 (0.0008) [2023-10-12 23:27:00,625][44959] Updated weights for policy 1, policy_version 81470 (0.0009) [2023-10-12 23:27:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166461440. Throughput: 0: 1630.0, 1: 1654.8. Samples: 41621114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:27:01,443][43579] Avg episode reward: [(0, '272.000'), (1, '275.480')] [2023-10-12 23:27:04,648][44959] Updated weights for policy 1, policy_version 81480 (0.0008) [2023-10-12 23:27:04,657][44958] Updated weights for policy 0, policy_version 81090 (0.0009) [2023-10-12 23:27:05,012][44959] Updated weights for policy 1, policy_version 81490 (0.0009) [2023-10-12 23:27:05,028][44958] Updated weights for policy 0, policy_version 81100 (0.0010) [2023-10-12 23:27:05,377][44959] Updated weights for policy 1, policy_version 81500 (0.0009) [2023-10-12 23:27:05,398][44958] Updated weights for policy 0, policy_version 81110 (0.0007) [2023-10-12 23:27:05,776][44958] Updated weights for policy 0, policy_version 81120 (0.0009) [2023-10-12 23:27:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 166526976. Throughput: 0: 1626.8, 1: 1657.2. Samples: 41632514. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:06,443][43579] Avg episode reward: [(0, '274.660'), (1, '279.710')] [2023-10-12 23:27:09,426][44959] Updated weights for policy 1, policy_version 81510 (0.0010) [2023-10-12 23:27:09,794][44959] Updated weights for policy 1, policy_version 81520 (0.0009) [2023-10-12 23:27:09,829][44958] Updated weights for policy 0, policy_version 81130 (0.0007) [2023-10-12 23:27:10,166][44959] Updated weights for policy 1, policy_version 81530 (0.0008) [2023-10-12 23:27:10,207][44958] Updated weights for policy 0, policy_version 81140 (0.0007) [2023-10-12 23:27:10,577][44958] Updated weights for policy 0, policy_version 81150 (0.0010) [2023-10-12 23:27:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166592512. Throughput: 0: 1632.4, 1: 1643.5. Samples: 41651064. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:11,444][43579] Avg episode reward: [(0, '274.980'), (1, '278.790')] [2023-10-12 23:27:14,387][44959] Updated weights for policy 1, policy_version 81540 (0.0008) [2023-10-12 23:27:14,765][44959] Updated weights for policy 1, policy_version 81550 (0.0008) [2023-10-12 23:27:14,783][44958] Updated weights for policy 0, policy_version 81160 (0.0008) [2023-10-12 23:27:15,135][44959] Updated weights for policy 1, policy_version 81560 (0.0008) [2023-10-12 23:27:15,148][44958] Updated weights for policy 0, policy_version 81170 (0.0007) [2023-10-12 23:27:15,526][44958] Updated weights for policy 0, policy_version 81180 (0.0008) [2023-10-12 23:27:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166658048. Throughput: 0: 1630.7, 1: 1658.3. Samples: 41670562. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:16,443][43579] Avg episode reward: [(0, '276.540'), (1, '274.420')] [2023-10-12 23:27:19,206][44959] Updated weights for policy 1, policy_version 81570 (0.0007) [2023-10-12 23:27:19,579][44959] Updated weights for policy 1, policy_version 81580 (0.0008) [2023-10-12 23:27:19,677][44958] Updated weights for policy 0, policy_version 81190 (0.0007) [2023-10-12 23:27:19,937][44959] Updated weights for policy 1, policy_version 81590 (0.0008) [2023-10-12 23:27:20,053][44958] Updated weights for policy 0, policy_version 81200 (0.0008) [2023-10-12 23:27:20,314][44959] Updated weights for policy 1, policy_version 81600 (0.0008) [2023-10-12 23:27:20,423][44958] Updated weights for policy 0, policy_version 81210 (0.0008) [2023-10-12 23:27:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166723584. Throughput: 0: 1632.4, 1: 1652.4. Samples: 41681942. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:21,443][43579] Avg episode reward: [(0, '275.810'), (1, '276.420')] [2023-10-12 23:27:24,359][44959] Updated weights for policy 1, policy_version 81610 (0.0008) [2023-10-12 23:27:24,650][44958] Updated weights for policy 0, policy_version 81220 (0.0010) [2023-10-12 23:27:24,725][44959] Updated weights for policy 1, policy_version 81620 (0.0008) [2023-10-12 23:27:25,020][44958] Updated weights for policy 0, policy_version 81230 (0.0009) [2023-10-12 23:27:25,090][44959] Updated weights for policy 1, policy_version 81630 (0.0007) [2023-10-12 23:27:25,397][44958] Updated weights for policy 0, policy_version 81240 (0.0010) [2023-10-12 23:27:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166789120. Throughput: 0: 1639.4, 1: 1647.9. Samples: 41700610. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:26,443][43579] Avg episode reward: [(0, '268.790'), (1, '278.220')] [2023-10-12 23:27:29,165][44959] Updated weights for policy 1, policy_version 81640 (0.0010) [2023-10-12 23:27:29,543][44959] Updated weights for policy 1, policy_version 81650 (0.0010) [2023-10-12 23:27:29,562][44958] Updated weights for policy 0, policy_version 81250 (0.0008) [2023-10-12 23:27:29,902][44959] Updated weights for policy 1, policy_version 81660 (0.0009) [2023-10-12 23:27:29,930][44958] Updated weights for policy 0, policy_version 81260 (0.0008) [2023-10-12 23:27:30,296][44958] Updated weights for policy 0, policy_version 81270 (0.0008) [2023-10-12 23:27:30,670][44958] Updated weights for policy 0, policy_version 81280 (0.0009) [2023-10-12 23:27:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166854656. Throughput: 0: 1639.4, 1: 1658.2. Samples: 41720224. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:31,443][43579] Avg episode reward: [(0, '265.960'), (1, '273.010')] [2023-10-12 23:27:34,212][44959] Updated weights for policy 1, policy_version 81670 (0.0010) [2023-10-12 23:27:34,597][44959] Updated weights for policy 1, policy_version 81680 (0.0009) [2023-10-12 23:27:34,863][44958] Updated weights for policy 0, policy_version 81290 (0.0010) [2023-10-12 23:27:34,965][44959] Updated weights for policy 1, policy_version 81690 (0.0008) [2023-10-12 23:27:35,238][44958] Updated weights for policy 0, policy_version 81300 (0.0007) [2023-10-12 23:27:35,610][44958] Updated weights for policy 0, policy_version 81310 (0.0010) [2023-10-12 23:27:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166920192. Throughput: 0: 1640.7, 1: 1656.7. Samples: 41731566. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:36,444][43579] Avg episode reward: [(0, '265.310'), (1, '268.320')] [2023-10-12 23:27:39,107][44959] Updated weights for policy 1, policy_version 81700 (0.0008) [2023-10-12 23:27:39,469][44959] Updated weights for policy 1, policy_version 81710 (0.0009) [2023-10-12 23:27:39,747][44958] Updated weights for policy 0, policy_version 81320 (0.0008) [2023-10-12 23:27:39,851][44959] Updated weights for policy 1, policy_version 81720 (0.0009) [2023-10-12 23:27:40,120][44958] Updated weights for policy 0, policy_version 81330 (0.0007) [2023-10-12 23:27:40,493][44958] Updated weights for policy 0, policy_version 81340 (0.0010) [2023-10-12 23:27:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 166985728. Throughput: 0: 1643.4, 1: 1647.0. Samples: 41750184. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:41,444][43579] Avg episode reward: [(0, '271.930'), (1, '275.140')] [2023-10-12 23:27:44,030][44959] Updated weights for policy 1, policy_version 81730 (0.0008) [2023-10-12 23:27:44,408][44959] Updated weights for policy 1, policy_version 81740 (0.0008) [2023-10-12 23:27:44,764][44959] Updated weights for policy 1, policy_version 81750 (0.0007) [2023-10-12 23:27:44,891][44958] Updated weights for policy 0, policy_version 81350 (0.0009) [2023-10-12 23:27:45,132][44959] Updated weights for policy 1, policy_version 81760 (0.0008) [2023-10-12 23:27:45,275][44958] Updated weights for policy 0, policy_version 81360 (0.0009) [2023-10-12 23:27:45,632][44958] Updated weights for policy 0, policy_version 81370 (0.0009) [2023-10-12 23:27:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 167051264. Throughput: 0: 1638.4, 1: 1659.5. Samples: 41769520. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:46,443][43579] Avg episode reward: [(0, '273.480'), (1, '276.760')] [2023-10-12 23:27:46,450][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000081760_83722240.pth... [2023-10-12 23:27:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000081376_83329024.pth... [2023-10-12 23:27:46,492][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000080224_82149376.pth [2023-10-12 23:27:46,498][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000079840_81756160.pth [2023-10-12 23:27:49,316][44959] Updated weights for policy 1, policy_version 81770 (0.0009) [2023-10-12 23:27:49,600][44958] Updated weights for policy 0, policy_version 81380 (0.0010) [2023-10-12 23:27:49,690][44959] Updated weights for policy 1, policy_version 81780 (0.0008) [2023-10-12 23:27:49,967][44958] Updated weights for policy 0, policy_version 81390 (0.0009) [2023-10-12 23:27:50,043][44959] Updated weights for policy 1, policy_version 81790 (0.0008) [2023-10-12 23:27:50,334][44958] Updated weights for policy 0, policy_version 81400 (0.0008) [2023-10-12 23:27:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 167116800. Throughput: 0: 1640.6, 1: 1648.4. Samples: 41780518. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:51,443][43579] Avg episode reward: [(0, '270.880'), (1, '281.100')] [2023-10-12 23:27:54,480][44959] Updated weights for policy 1, policy_version 81800 (0.0007) [2023-10-12 23:27:54,570][44958] Updated weights for policy 0, policy_version 81410 (0.0008) [2023-10-12 23:27:54,843][44959] Updated weights for policy 1, policy_version 81810 (0.0007) [2023-10-12 23:27:54,943][44958] Updated weights for policy 0, policy_version 81420 (0.0008) [2023-10-12 23:27:55,210][44959] Updated weights for policy 1, policy_version 81820 (0.0008) [2023-10-12 23:27:55,315][44958] Updated weights for policy 0, policy_version 81430 (0.0008) [2023-10-12 23:27:55,683][44958] Updated weights for policy 0, policy_version 81440 (0.0009) [2023-10-12 23:27:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 167182336. Throughput: 0: 1640.6, 1: 1654.7. Samples: 41799352. Policy #0 lag: (min: 27.0, avg: 40.0, max: 59.0) [2023-10-12 23:27:56,444][43579] Avg episode reward: [(0, '280.040'), (1, '282.080')] [2023-10-12 23:27:59,277][44959] Updated weights for policy 1, policy_version 81830 (0.0010) [2023-10-12 23:27:59,651][44959] Updated weights for policy 1, policy_version 81840 (0.0008) [2023-10-12 23:27:59,874][44958] Updated weights for policy 0, policy_version 81450 (0.0007) [2023-10-12 23:28:00,012][44959] Updated weights for policy 1, policy_version 81850 (0.0008) [2023-10-12 23:28:00,253][44958] Updated weights for policy 0, policy_version 81460 (0.0009) [2023-10-12 23:28:00,613][44958] Updated weights for policy 0, policy_version 81470 (0.0010) [2023-10-12 23:28:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 167247872. Throughput: 0: 1636.4, 1: 1652.8. Samples: 41818580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:01,444][43579] Avg episode reward: [(0, '285.730'), (1, '282.270')] [2023-10-12 23:28:04,126][44959] Updated weights for policy 1, policy_version 81860 (0.0008) [2023-10-12 23:28:04,497][44959] Updated weights for policy 1, policy_version 81870 (0.0009) [2023-10-12 23:28:04,866][44959] Updated weights for policy 1, policy_version 81880 (0.0007) [2023-10-12 23:28:04,948][44958] Updated weights for policy 0, policy_version 81480 (0.0009) [2023-10-12 23:28:05,313][44958] Updated weights for policy 0, policy_version 81490 (0.0008) [2023-10-12 23:28:05,692][44958] Updated weights for policy 0, policy_version 81500 (0.0008) [2023-10-12 23:28:06,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 167313408. Throughput: 0: 1635.0, 1: 1648.0. Samples: 41829676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:06,443][43579] Avg episode reward: [(0, '284.110'), (1, '280.950')] [2023-10-12 23:28:08,931][44959] Updated weights for policy 1, policy_version 81890 (0.0007) [2023-10-12 23:28:09,301][44959] Updated weights for policy 1, policy_version 81900 (0.0008) [2023-10-12 23:28:09,662][44959] Updated weights for policy 1, policy_version 81910 (0.0008) [2023-10-12 23:28:09,720][44958] Updated weights for policy 0, policy_version 81510 (0.0007) [2023-10-12 23:28:10,032][44959] Updated weights for policy 1, policy_version 81920 (0.0008) [2023-10-12 23:28:10,083][44958] Updated weights for policy 0, policy_version 81520 (0.0009) [2023-10-12 23:28:10,464][44958] Updated weights for policy 0, policy_version 81530 (0.0007) [2023-10-12 23:28:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 167378944. Throughput: 0: 1632.8, 1: 1648.3. Samples: 41848262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:11,443][43579] Avg episode reward: [(0, '284.750'), (1, '281.960')] [2023-10-12 23:28:14,033][44959] Updated weights for policy 1, policy_version 81930 (0.0007) [2023-10-12 23:28:14,406][44959] Updated weights for policy 1, policy_version 81940 (0.0007) [2023-10-12 23:28:14,589][44958] Updated weights for policy 0, policy_version 81540 (0.0009) [2023-10-12 23:28:14,782][44959] Updated weights for policy 1, policy_version 81950 (0.0009) [2023-10-12 23:28:14,963][44958] Updated weights for policy 0, policy_version 81550 (0.0009) [2023-10-12 23:28:15,330][44958] Updated weights for policy 0, policy_version 81560 (0.0008) [2023-10-12 23:28:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 167444480. Throughput: 0: 1630.6, 1: 1650.4. Samples: 41867870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:16,443][43579] Avg episode reward: [(0, '285.550'), (1, '277.130')] [2023-10-12 23:28:18,996][44959] Updated weights for policy 1, policy_version 81960 (0.0008) [2023-10-12 23:28:19,362][44959] Updated weights for policy 1, policy_version 81970 (0.0009) [2023-10-12 23:28:19,578][44958] Updated weights for policy 0, policy_version 81570 (0.0009) [2023-10-12 23:28:19,735][44959] Updated weights for policy 1, policy_version 81980 (0.0007) [2023-10-12 23:28:19,949][44958] Updated weights for policy 0, policy_version 81580 (0.0009) [2023-10-12 23:28:20,323][44958] Updated weights for policy 0, policy_version 81590 (0.0009) [2023-10-12 23:28:20,701][44958] Updated weights for policy 0, policy_version 81600 (0.0012) [2023-10-12 23:28:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 167510016. Throughput: 0: 1628.5, 1: 1643.2. Samples: 41878792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:21,443][43579] Avg episode reward: [(0, '289.200'), (1, '273.880')] [2023-10-12 23:28:23,897][44959] Updated weights for policy 1, policy_version 81990 (0.0008) [2023-10-12 23:28:24,273][44959] Updated weights for policy 1, policy_version 82000 (0.0009) [2023-10-12 23:28:24,639][44959] Updated weights for policy 1, policy_version 82010 (0.0007) [2023-10-12 23:28:25,101][44958] Updated weights for policy 0, policy_version 81610 (0.0008) [2023-10-12 23:28:25,470][44958] Updated weights for policy 0, policy_version 81620 (0.0010) [2023-10-12 23:28:25,838][44958] Updated weights for policy 0, policy_version 81630 (0.0008) [2023-10-12 23:28:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 167575552. Throughput: 0: 1628.1, 1: 1647.2. Samples: 41897572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:26,443][43579] Avg episode reward: [(0, '284.890'), (1, '276.620')] [2023-10-12 23:28:28,886][44959] Updated weights for policy 1, policy_version 82020 (0.0008) [2023-10-12 23:28:29,255][44959] Updated weights for policy 1, policy_version 82030 (0.0009) [2023-10-12 23:28:29,623][44959] Updated weights for policy 1, policy_version 82040 (0.0008) [2023-10-12 23:28:30,051][44958] Updated weights for policy 0, policy_version 81640 (0.0007) [2023-10-12 23:28:30,427][44958] Updated weights for policy 0, policy_version 81650 (0.0008) [2023-10-12 23:28:30,799][44958] Updated weights for policy 0, policy_version 81660 (0.0009) [2023-10-12 23:28:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 167641088. Throughput: 0: 1628.9, 1: 1647.2. Samples: 41916942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:31,443][43579] Avg episode reward: [(0, '284.510'), (1, '270.220')] [2023-10-12 23:28:33,900][44959] Updated weights for policy 1, policy_version 82050 (0.0009) [2023-10-12 23:28:34,306][44959] Updated weights for policy 1, policy_version 82060 (0.0009) [2023-10-12 23:28:34,673][44959] Updated weights for policy 1, policy_version 82070 (0.0008) [2023-10-12 23:28:34,857][44958] Updated weights for policy 0, policy_version 81670 (0.0008) [2023-10-12 23:28:35,043][44959] Updated weights for policy 1, policy_version 82080 (0.0007) [2023-10-12 23:28:35,229][44958] Updated weights for policy 0, policy_version 81680 (0.0008) [2023-10-12 23:28:35,594][44958] Updated weights for policy 0, policy_version 81690 (0.0010) [2023-10-12 23:28:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 167706624. Throughput: 0: 1626.8, 1: 1649.1. Samples: 41927938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:36,444][43579] Avg episode reward: [(0, '280.200'), (1, '268.930')] [2023-10-12 23:28:39,207][44959] Updated weights for policy 1, policy_version 82090 (0.0009) [2023-10-12 23:28:39,582][44959] Updated weights for policy 1, policy_version 82100 (0.0010) [2023-10-12 23:28:39,847][44958] Updated weights for policy 0, policy_version 81700 (0.0010) [2023-10-12 23:28:39,953][44959] Updated weights for policy 1, policy_version 82110 (0.0007) [2023-10-12 23:28:40,213][44958] Updated weights for policy 0, policy_version 81710 (0.0007) [2023-10-12 23:28:40,594][44958] Updated weights for policy 0, policy_version 81720 (0.0007) [2023-10-12 23:28:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 167772160. Throughput: 0: 1632.1, 1: 1640.5. Samples: 41946618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:41,443][43579] Avg episode reward: [(0, '280.670'), (1, '265.040')] [2023-10-12 23:28:44,232][44959] Updated weights for policy 1, policy_version 82120 (0.0008) [2023-10-12 23:28:44,607][44959] Updated weights for policy 1, policy_version 82130 (0.0007) [2023-10-12 23:28:44,803][44958] Updated weights for policy 0, policy_version 81730 (0.0007) [2023-10-12 23:28:44,964][44959] Updated weights for policy 1, policy_version 82140 (0.0007) [2023-10-12 23:28:45,178][44958] Updated weights for policy 0, policy_version 81740 (0.0008) [2023-10-12 23:28:45,548][44958] Updated weights for policy 0, policy_version 81750 (0.0008) [2023-10-12 23:28:45,914][44958] Updated weights for policy 0, policy_version 81760 (0.0007) [2023-10-12 23:28:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 167837696. Throughput: 0: 1628.1, 1: 1645.1. Samples: 41965876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:46,443][43579] Avg episode reward: [(0, '278.610'), (1, '264.550')] [2023-10-12 23:28:48,929][44959] Updated weights for policy 1, policy_version 82150 (0.0008) [2023-10-12 23:28:49,294][44959] Updated weights for policy 1, policy_version 82160 (0.0008) [2023-10-12 23:28:49,661][44959] Updated weights for policy 1, policy_version 82170 (0.0009) [2023-10-12 23:28:50,176][44958] Updated weights for policy 0, policy_version 81770 (0.0010) [2023-10-12 23:28:50,550][44958] Updated weights for policy 0, policy_version 81780 (0.0008) [2023-10-12 23:28:50,922][44958] Updated weights for policy 0, policy_version 81790 (0.0007) [2023-10-12 23:28:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 167903232. Throughput: 0: 1625.6, 1: 1644.7. Samples: 41976838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:51,443][43579] Avg episode reward: [(0, '275.460'), (1, '264.290')] [2023-10-12 23:28:53,729][44959] Updated weights for policy 1, policy_version 82180 (0.0009) [2023-10-12 23:28:54,096][44959] Updated weights for policy 1, policy_version 82190 (0.0010) [2023-10-12 23:28:54,473][44959] Updated weights for policy 1, policy_version 82200 (0.0008) [2023-10-12 23:28:55,111][44958] Updated weights for policy 0, policy_version 81800 (0.0008) [2023-10-12 23:28:55,481][44958] Updated weights for policy 0, policy_version 81810 (0.0009) [2023-10-12 23:28:55,864][44958] Updated weights for policy 0, policy_version 81820 (0.0007) [2023-10-12 23:28:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 167968768. Throughput: 0: 1632.9, 1: 1652.4. Samples: 41996102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:28:56,444][43579] Avg episode reward: [(0, '277.900'), (1, '269.140')] [2023-10-12 23:28:58,655][44959] Updated weights for policy 1, policy_version 82210 (0.0008) [2023-10-12 23:28:59,023][44959] Updated weights for policy 1, policy_version 82220 (0.0009) [2023-10-12 23:28:59,392][44959] Updated weights for policy 1, policy_version 82230 (0.0008) [2023-10-12 23:28:59,761][44959] Updated weights for policy 1, policy_version 82240 (0.0007) [2023-10-12 23:28:59,863][44958] Updated weights for policy 0, policy_version 81830 (0.0009) [2023-10-12 23:29:00,237][44958] Updated weights for policy 0, policy_version 81840 (0.0008) [2023-10-12 23:29:00,605][44958] Updated weights for policy 0, policy_version 81850 (0.0007) [2023-10-12 23:29:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168034304. Throughput: 0: 1629.6, 1: 1649.7. Samples: 42015442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:01,443][43579] Avg episode reward: [(0, '279.860'), (1, '272.880')] [2023-10-12 23:29:03,820][44959] Updated weights for policy 1, policy_version 82250 (0.0008) [2023-10-12 23:29:04,189][44959] Updated weights for policy 1, policy_version 82260 (0.0007) [2023-10-12 23:29:04,558][44959] Updated weights for policy 1, policy_version 82270 (0.0008) [2023-10-12 23:29:04,736][44958] Updated weights for policy 0, policy_version 81860 (0.0007) [2023-10-12 23:29:05,113][44958] Updated weights for policy 0, policy_version 81870 (0.0008) [2023-10-12 23:29:05,489][44958] Updated weights for policy 0, policy_version 81880 (0.0009) [2023-10-12 23:29:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168099840. Throughput: 0: 1630.5, 1: 1647.5. Samples: 42026302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:06,443][43579] Avg episode reward: [(0, '281.350'), (1, '279.170')] [2023-10-12 23:29:08,705][44959] Updated weights for policy 1, policy_version 82280 (0.0008) [2023-10-12 23:29:09,082][44959] Updated weights for policy 1, policy_version 82290 (0.0009) [2023-10-12 23:29:09,458][44959] Updated weights for policy 1, policy_version 82300 (0.0008) [2023-10-12 23:29:09,833][44958] Updated weights for policy 0, policy_version 81890 (0.0008) [2023-10-12 23:29:10,199][44958] Updated weights for policy 0, policy_version 81900 (0.0009) [2023-10-12 23:29:10,568][44958] Updated weights for policy 0, policy_version 81910 (0.0008) [2023-10-12 23:29:10,936][44958] Updated weights for policy 0, policy_version 81920 (0.0008) [2023-10-12 23:29:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168165376. Throughput: 0: 1634.8, 1: 1653.7. Samples: 42045558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:11,444][43579] Avg episode reward: [(0, '279.970'), (1, '278.440')] [2023-10-12 23:29:13,542][44959] Updated weights for policy 1, policy_version 82310 (0.0010) [2023-10-12 23:29:13,920][44959] Updated weights for policy 1, policy_version 82320 (0.0009) [2023-10-12 23:29:14,294][44959] Updated weights for policy 1, policy_version 82330 (0.0007) [2023-10-12 23:29:15,141][44958] Updated weights for policy 0, policy_version 81930 (0.0010) [2023-10-12 23:29:15,512][44958] Updated weights for policy 0, policy_version 81940 (0.0011) [2023-10-12 23:29:15,879][44958] Updated weights for policy 0, policy_version 81950 (0.0010) [2023-10-12 23:29:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168230912. Throughput: 0: 1632.0, 1: 1661.4. Samples: 42065144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:16,443][43579] Avg episode reward: [(0, '285.980'), (1, '277.850')] [2023-10-12 23:29:18,290][44959] Updated weights for policy 1, policy_version 82340 (0.0008) [2023-10-12 23:29:18,681][44959] Updated weights for policy 1, policy_version 82350 (0.0007) [2023-10-12 23:29:19,048][44959] Updated weights for policy 1, policy_version 82360 (0.0008) [2023-10-12 23:29:19,989][44958] Updated weights for policy 0, policy_version 81960 (0.0008) [2023-10-12 23:29:20,355][44958] Updated weights for policy 0, policy_version 81970 (0.0007) [2023-10-12 23:29:20,735][44958] Updated weights for policy 0, policy_version 81980 (0.0007) [2023-10-12 23:29:21,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168296448. Throughput: 0: 1633.3, 1: 1649.9. Samples: 42075684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:21,444][43579] Avg episode reward: [(0, '284.810'), (1, '281.470')] [2023-10-12 23:29:23,039][44959] Updated weights for policy 1, policy_version 82370 (0.0009) [2023-10-12 23:29:23,403][44959] Updated weights for policy 1, policy_version 82380 (0.0009) [2023-10-12 23:29:23,768][44959] Updated weights for policy 1, policy_version 82390 (0.0009) [2023-10-12 23:29:24,139][44959] Updated weights for policy 1, policy_version 82400 (0.0008) [2023-10-12 23:29:24,816][44958] Updated weights for policy 0, policy_version 81990 (0.0008) [2023-10-12 23:29:25,190][44958] Updated weights for policy 0, policy_version 82000 (0.0011) [2023-10-12 23:29:25,576][44958] Updated weights for policy 0, policy_version 82010 (0.0010) [2023-10-12 23:29:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168361984. Throughput: 0: 1636.3, 1: 1669.7. Samples: 42095390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:26,444][43579] Avg episode reward: [(0, '280.640'), (1, '277.320')] [2023-10-12 23:29:28,046][44959] Updated weights for policy 1, policy_version 82410 (0.0009) [2023-10-12 23:29:28,415][44959] Updated weights for policy 1, policy_version 82420 (0.0010) [2023-10-12 23:29:28,784][44959] Updated weights for policy 1, policy_version 82430 (0.0011) [2023-10-12 23:29:30,021][44958] Updated weights for policy 0, policy_version 82020 (0.0011) [2023-10-12 23:29:30,395][44958] Updated weights for policy 0, policy_version 82030 (0.0007) [2023-10-12 23:29:30,771][44958] Updated weights for policy 0, policy_version 82040 (0.0008) [2023-10-12 23:29:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168427520. Throughput: 0: 1639.1, 1: 1681.8. Samples: 42115318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:31,444][43579] Avg episode reward: [(0, '274.960'), (1, '274.060')] [2023-10-12 23:29:32,962][44959] Updated weights for policy 1, policy_version 82440 (0.0010) [2023-10-12 23:29:33,321][44959] Updated weights for policy 1, policy_version 82450 (0.0010) [2023-10-12 23:29:33,694][44959] Updated weights for policy 1, policy_version 82460 (0.0007) [2023-10-12 23:29:34,660][44958] Updated weights for policy 0, policy_version 82050 (0.0010) [2023-10-12 23:29:35,031][44958] Updated weights for policy 0, policy_version 82060 (0.0009) [2023-10-12 23:29:35,412][44958] Updated weights for policy 0, policy_version 82070 (0.0008) [2023-10-12 23:29:35,771][44958] Updated weights for policy 0, policy_version 82080 (0.0011) [2023-10-12 23:29:36,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168493056. Throughput: 0: 1646.9, 1: 1658.4. Samples: 42125578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:36,444][43579] Avg episode reward: [(0, '273.040'), (1, '276.840')] [2023-10-12 23:29:37,796][44959] Updated weights for policy 1, policy_version 82470 (0.0008) [2023-10-12 23:29:38,158][44959] Updated weights for policy 1, policy_version 82480 (0.0009) [2023-10-12 23:29:38,523][44959] Updated weights for policy 1, policy_version 82490 (0.0007) [2023-10-12 23:29:39,927][44958] Updated weights for policy 0, policy_version 82090 (0.0009) [2023-10-12 23:29:40,302][44958] Updated weights for policy 0, policy_version 82100 (0.0007) [2023-10-12 23:29:40,667][44958] Updated weights for policy 0, policy_version 82110 (0.0007) [2023-10-12 23:29:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168558592. Throughput: 0: 1643.9, 1: 1669.4. Samples: 42145200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:41,444][43579] Avg episode reward: [(0, '271.910'), (1, '275.950')] [2023-10-12 23:29:42,898][44959] Updated weights for policy 1, policy_version 82500 (0.0007) [2023-10-12 23:29:43,271][44959] Updated weights for policy 1, policy_version 82510 (0.0010) [2023-10-12 23:29:43,645][44959] Updated weights for policy 1, policy_version 82520 (0.0009) [2023-10-12 23:29:44,995][44958] Updated weights for policy 0, policy_version 82120 (0.0010) [2023-10-12 23:29:45,373][44958] Updated weights for policy 0, policy_version 82130 (0.0010) [2023-10-12 23:29:45,746][44958] Updated weights for policy 0, policy_version 82140 (0.0010) [2023-10-12 23:29:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168624128. Throughput: 0: 1640.9, 1: 1674.9. Samples: 42164656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:29:46,443][43579] Avg episode reward: [(0, '269.690'), (1, '281.110')] [2023-10-12 23:29:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000082528_84508672.pth... [2023-10-12 23:29:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000082144_84115456.pth... [2023-10-12 23:29:46,482][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000080992_82935808.pth [2023-10-12 23:29:46,483][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000080608_82542592.pth [2023-10-12 23:29:47,742][44959] Updated weights for policy 1, policy_version 82530 (0.0008) [2023-10-12 23:29:48,106][44959] Updated weights for policy 1, policy_version 82540 (0.0009) [2023-10-12 23:29:48,481][44959] Updated weights for policy 1, policy_version 82550 (0.0007) [2023-10-12 23:29:48,846][44959] Updated weights for policy 1, policy_version 82560 (0.0007) [2023-10-12 23:29:50,012][44958] Updated weights for policy 0, policy_version 82150 (0.0010) [2023-10-12 23:29:50,373][44958] Updated weights for policy 0, policy_version 82160 (0.0008) [2023-10-12 23:29:50,744][44958] Updated weights for policy 0, policy_version 82170 (0.0008) [2023-10-12 23:29:51,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168689664. Throughput: 0: 1640.5, 1: 1658.2. Samples: 42174744. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:29:51,444][43579] Avg episode reward: [(0, '268.350'), (1, '276.690')] [2023-10-12 23:29:52,963][44959] Updated weights for policy 1, policy_version 82570 (0.0008) [2023-10-12 23:29:53,337][44959] Updated weights for policy 1, policy_version 82580 (0.0010) [2023-10-12 23:29:53,702][44959] Updated weights for policy 1, policy_version 82590 (0.0008) [2023-10-12 23:29:55,189][44958] Updated weights for policy 0, policy_version 82180 (0.0008) [2023-10-12 23:29:55,558][44958] Updated weights for policy 0, policy_version 82190 (0.0007) [2023-10-12 23:29:55,927][44958] Updated weights for policy 0, policy_version 82200 (0.0011) [2023-10-12 23:29:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168755200. Throughput: 0: 1646.0, 1: 1669.4. Samples: 42194752. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:29:56,443][43579] Avg episode reward: [(0, '259.530'), (1, '283.010')] [2023-10-12 23:29:57,862][44959] Updated weights for policy 1, policy_version 82600 (0.0009) [2023-10-12 23:29:58,231][44959] Updated weights for policy 1, policy_version 82610 (0.0008) [2023-10-12 23:29:58,598][44959] Updated weights for policy 1, policy_version 82620 (0.0009) [2023-10-12 23:30:00,082][44958] Updated weights for policy 0, policy_version 82210 (0.0010) [2023-10-12 23:30:00,489][44958] Updated weights for policy 0, policy_version 82220 (0.0010) [2023-10-12 23:30:00,859][44958] Updated weights for policy 0, policy_version 82230 (0.0007) [2023-10-12 23:30:01,224][44958] Updated weights for policy 0, policy_version 82240 (0.0007) [2023-10-12 23:30:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168820736. Throughput: 0: 1644.3, 1: 1670.2. Samples: 42214298. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:30:01,443][43579] Avg episode reward: [(0, '266.040'), (1, '282.040')] [2023-10-12 23:30:02,608][44959] Updated weights for policy 1, policy_version 82630 (0.0009) [2023-10-12 23:30:02,979][44959] Updated weights for policy 1, policy_version 82640 (0.0008) [2023-10-12 23:30:03,342][44959] Updated weights for policy 1, policy_version 82650 (0.0008) [2023-10-12 23:30:05,165][44958] Updated weights for policy 0, policy_version 82250 (0.0008) [2023-10-12 23:30:05,540][44958] Updated weights for policy 0, policy_version 82260 (0.0008) [2023-10-12 23:30:05,916][44958] Updated weights for policy 0, policy_version 82270 (0.0007) [2023-10-12 23:30:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168886272. Throughput: 0: 1642.1, 1: 1661.0. Samples: 42224324. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:30:06,443][43579] Avg episode reward: [(0, '264.890'), (1, '283.590')] [2023-10-12 23:30:07,449][44959] Updated weights for policy 1, policy_version 82660 (0.0007) [2023-10-12 23:30:07,859][44959] Updated weights for policy 1, policy_version 82670 (0.0007) [2023-10-12 23:30:08,221][44959] Updated weights for policy 1, policy_version 82680 (0.0007) [2023-10-12 23:30:10,125][44958] Updated weights for policy 0, policy_version 82280 (0.0007) [2023-10-12 23:30:10,500][44958] Updated weights for policy 0, policy_version 82290 (0.0009) [2023-10-12 23:30:10,868][44958] Updated weights for policy 0, policy_version 82300 (0.0009) [2023-10-12 23:30:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 168951808. Throughput: 0: 1641.4, 1: 1668.0. Samples: 42244312. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:30:11,443][43579] Avg episode reward: [(0, '270.850'), (1, '282.400')] [2023-10-12 23:30:12,335][44959] Updated weights for policy 1, policy_version 82690 (0.0007) [2023-10-12 23:30:12,713][44959] Updated weights for policy 1, policy_version 82700 (0.0007) [2023-10-12 23:30:13,073][44959] Updated weights for policy 1, policy_version 82710 (0.0010) [2023-10-12 23:30:13,446][44959] Updated weights for policy 1, policy_version 82720 (0.0009) [2023-10-12 23:30:15,097][44958] Updated weights for policy 0, policy_version 82310 (0.0008) [2023-10-12 23:30:15,471][44958] Updated weights for policy 0, policy_version 82320 (0.0008) [2023-10-12 23:30:15,835][44958] Updated weights for policy 0, policy_version 82330 (0.0011) [2023-10-12 23:30:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169017344. Throughput: 0: 1637.7, 1: 1659.1. Samples: 42263674. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:30:16,444][43579] Avg episode reward: [(0, '269.630'), (1, '286.500')] [2023-10-12 23:30:17,430][44959] Updated weights for policy 1, policy_version 82730 (0.0008) [2023-10-12 23:30:17,798][44959] Updated weights for policy 1, policy_version 82740 (0.0007) [2023-10-12 23:30:18,168][44959] Updated weights for policy 1, policy_version 82750 (0.0008) [2023-10-12 23:30:19,804][44958] Updated weights for policy 0, policy_version 82340 (0.0010) [2023-10-12 23:30:20,176][44958] Updated weights for policy 0, policy_version 82350 (0.0008) [2023-10-12 23:30:20,540][44958] Updated weights for policy 0, policy_version 82360 (0.0007) [2023-10-12 23:30:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169082880. Throughput: 0: 1640.0, 1: 1659.5. Samples: 42274056. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:30:21,443][43579] Avg episode reward: [(0, '270.250'), (1, '289.400')] [2023-10-12 23:30:22,403][44959] Updated weights for policy 1, policy_version 82760 (0.0008) [2023-10-12 23:30:22,775][44959] Updated weights for policy 1, policy_version 82770 (0.0007) [2023-10-12 23:30:23,138][44959] Updated weights for policy 1, policy_version 82780 (0.0009) [2023-10-12 23:30:24,788][44958] Updated weights for policy 0, policy_version 82370 (0.0008) [2023-10-12 23:30:25,151][44958] Updated weights for policy 0, policy_version 82380 (0.0009) [2023-10-12 23:30:25,534][44958] Updated weights for policy 0, policy_version 82390 (0.0007) [2023-10-12 23:30:25,901][44958] Updated weights for policy 0, policy_version 82400 (0.0007) [2023-10-12 23:30:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169148416. Throughput: 0: 1633.2, 1: 1666.9. Samples: 42293704. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:30:26,444][43579] Avg episode reward: [(0, '282.720'), (1, '286.780')] [2023-10-12 23:30:27,345][44959] Updated weights for policy 1, policy_version 82790 (0.0007) [2023-10-12 23:30:27,707][44959] Updated weights for policy 1, policy_version 82800 (0.0008) [2023-10-12 23:30:28,078][44959] Updated weights for policy 1, policy_version 82810 (0.0008) [2023-10-12 23:30:30,231][44958] Updated weights for policy 0, policy_version 82410 (0.0008) [2023-10-12 23:30:30,609][44958] Updated weights for policy 0, policy_version 82420 (0.0009) [2023-10-12 23:30:30,981][44958] Updated weights for policy 0, policy_version 82430 (0.0009) [2023-10-12 23:30:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169213952. Throughput: 0: 1633.9, 1: 1661.2. Samples: 42312934. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:30:31,444][43579] Avg episode reward: [(0, '274.570'), (1, '281.610')] [2023-10-12 23:30:32,127][44959] Updated weights for policy 1, policy_version 82820 (0.0009) [2023-10-12 23:30:32,498][44959] Updated weights for policy 1, policy_version 82830 (0.0009) [2023-10-12 23:30:32,868][44959] Updated weights for policy 1, policy_version 82840 (0.0008) [2023-10-12 23:30:35,058][44958] Updated weights for policy 0, policy_version 82440 (0.0007) [2023-10-12 23:30:35,434][44958] Updated weights for policy 0, policy_version 82450 (0.0007) [2023-10-12 23:30:35,798][44958] Updated weights for policy 0, policy_version 82460 (0.0008) [2023-10-12 23:30:36,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169279488. Throughput: 0: 1636.3, 1: 1662.8. Samples: 42323200. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:30:36,443][43579] Avg episode reward: [(0, '273.670'), (1, '279.590')] [2023-10-12 23:30:37,103][44959] Updated weights for policy 1, policy_version 82850 (0.0009) [2023-10-12 23:30:37,471][44959] Updated weights for policy 1, policy_version 82860 (0.0009) [2023-10-12 23:30:37,839][44959] Updated weights for policy 1, policy_version 82870 (0.0010) [2023-10-12 23:30:38,205][44959] Updated weights for policy 1, policy_version 82880 (0.0009) [2023-10-12 23:30:39,832][44958] Updated weights for policy 0, policy_version 82470 (0.0010) [2023-10-12 23:30:40,207][44958] Updated weights for policy 0, policy_version 82480 (0.0010) [2023-10-12 23:30:40,590][44958] Updated weights for policy 0, policy_version 82490 (0.0010) [2023-10-12 23:30:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169345024. Throughput: 0: 1628.9, 1: 1660.2. Samples: 42342762. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-12 23:30:41,444][43579] Avg episode reward: [(0, '265.250'), (1, '280.370')] [2023-10-12 23:30:42,404][44959] Updated weights for policy 1, policy_version 82890 (0.0007) [2023-10-12 23:30:42,771][44959] Updated weights for policy 1, policy_version 82900 (0.0009) [2023-10-12 23:30:43,142][44959] Updated weights for policy 1, policy_version 82910 (0.0008) [2023-10-12 23:30:44,867][44958] Updated weights for policy 0, policy_version 82500 (0.0010) [2023-10-12 23:30:45,246][44958] Updated weights for policy 0, policy_version 82510 (0.0007) [2023-10-12 23:30:45,619][44958] Updated weights for policy 0, policy_version 82520 (0.0008) [2023-10-12 23:30:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169410560. Throughput: 0: 1629.5, 1: 1659.5. Samples: 42362300. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:30:46,443][43579] Avg episode reward: [(0, '266.710'), (1, '281.550')] [2023-10-12 23:30:47,301][44959] Updated weights for policy 1, policy_version 82920 (0.0011) [2023-10-12 23:30:47,668][44959] Updated weights for policy 1, policy_version 82930 (0.0008) [2023-10-12 23:30:48,034][44959] Updated weights for policy 1, policy_version 82940 (0.0011) [2023-10-12 23:30:49,715][44958] Updated weights for policy 0, policy_version 82530 (0.0008) [2023-10-12 23:30:50,116][44958] Updated weights for policy 0, policy_version 82540 (0.0008) [2023-10-12 23:30:50,484][44958] Updated weights for policy 0, policy_version 82550 (0.0007) [2023-10-12 23:30:50,864][44958] Updated weights for policy 0, policy_version 82560 (0.0008) [2023-10-12 23:30:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169476096. Throughput: 0: 1634.0, 1: 1660.0. Samples: 42372556. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:30:51,444][43579] Avg episode reward: [(0, '266.940'), (1, '282.550')] [2023-10-12 23:30:52,233][44959] Updated weights for policy 1, policy_version 82950 (0.0007) [2023-10-12 23:30:52,609][44959] Updated weights for policy 1, policy_version 82960 (0.0009) [2023-10-12 23:30:52,983][44959] Updated weights for policy 1, policy_version 82970 (0.0009) [2023-10-12 23:30:55,068][44958] Updated weights for policy 0, policy_version 82570 (0.0008) [2023-10-12 23:30:55,444][44958] Updated weights for policy 0, policy_version 82580 (0.0009) [2023-10-12 23:30:55,814][44958] Updated weights for policy 0, policy_version 82590 (0.0009) [2023-10-12 23:30:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169541632. Throughput: 0: 1628.8, 1: 1655.1. Samples: 42392088. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:30:56,444][43579] Avg episode reward: [(0, '264.820'), (1, '279.120')] [2023-10-12 23:30:57,034][44959] Updated weights for policy 1, policy_version 82980 (0.0008) [2023-10-12 23:30:57,401][44959] Updated weights for policy 1, policy_version 82990 (0.0008) [2023-10-12 23:30:57,769][44959] Updated weights for policy 1, policy_version 83000 (0.0009) [2023-10-12 23:31:00,039][44958] Updated weights for policy 0, policy_version 82600 (0.0009) [2023-10-12 23:31:00,419][44958] Updated weights for policy 0, policy_version 82610 (0.0007) [2023-10-12 23:31:00,779][44958] Updated weights for policy 0, policy_version 82620 (0.0010) [2023-10-12 23:31:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169607168. Throughput: 0: 1630.9, 1: 1656.6. Samples: 42411612. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:31:01,443][43579] Avg episode reward: [(0, '267.780'), (1, '285.160')] [2023-10-12 23:31:02,050][44959] Updated weights for policy 1, policy_version 83010 (0.0008) [2023-10-12 23:31:02,407][44959] Updated weights for policy 1, policy_version 83020 (0.0009) [2023-10-12 23:31:02,782][44959] Updated weights for policy 1, policy_version 83030 (0.0007) [2023-10-12 23:31:03,145][44959] Updated weights for policy 1, policy_version 83040 (0.0011) [2023-10-12 23:31:04,964][44958] Updated weights for policy 0, policy_version 82630 (0.0011) [2023-10-12 23:31:05,336][44958] Updated weights for policy 0, policy_version 82640 (0.0008) [2023-10-12 23:31:05,705][44958] Updated weights for policy 0, policy_version 82650 (0.0008) [2023-10-12 23:31:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169672704. Throughput: 0: 1625.5, 1: 1658.9. Samples: 42421856. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:31:06,444][43579] Avg episode reward: [(0, '270.130'), (1, '286.140')] [2023-10-12 23:31:07,438][44959] Updated weights for policy 1, policy_version 83050 (0.0007) [2023-10-12 23:31:07,805][44959] Updated weights for policy 1, policy_version 83060 (0.0008) [2023-10-12 23:31:08,181][44959] Updated weights for policy 1, policy_version 83070 (0.0010) [2023-10-12 23:31:09,918][44958] Updated weights for policy 0, policy_version 82660 (0.0008) [2023-10-12 23:31:10,293][44958] Updated weights for policy 0, policy_version 82670 (0.0009) [2023-10-12 23:31:10,670][44958] Updated weights for policy 0, policy_version 82680 (0.0010) [2023-10-12 23:31:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169738240. Throughput: 0: 1638.3, 1: 1650.9. Samples: 42441714. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:31:11,443][43579] Avg episode reward: [(0, '274.820'), (1, '282.000')] [2023-10-12 23:31:12,173][44959] Updated weights for policy 1, policy_version 83080 (0.0008) [2023-10-12 23:31:12,541][44959] Updated weights for policy 1, policy_version 83090 (0.0008) [2023-10-12 23:31:12,915][44959] Updated weights for policy 1, policy_version 83100 (0.0008) [2023-10-12 23:31:14,972][44958] Updated weights for policy 0, policy_version 82690 (0.0008) [2023-10-12 23:31:15,343][44958] Updated weights for policy 0, policy_version 82700 (0.0009) [2023-10-12 23:31:15,720][44958] Updated weights for policy 0, policy_version 82710 (0.0008) [2023-10-12 23:31:16,095][44958] Updated weights for policy 0, policy_version 82720 (0.0008) [2023-10-12 23:31:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169803776. Throughput: 0: 1639.6, 1: 1653.1. Samples: 42461102. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:31:16,443][43579] Avg episode reward: [(0, '270.240'), (1, '278.300')] [2023-10-12 23:31:17,012][44959] Updated weights for policy 1, policy_version 83110 (0.0008) [2023-10-12 23:31:17,394][44959] Updated weights for policy 1, policy_version 83120 (0.0010) [2023-10-12 23:31:17,764][44959] Updated weights for policy 1, policy_version 83130 (0.0009) [2023-10-12 23:31:20,256][44958] Updated weights for policy 0, policy_version 82730 (0.0008) [2023-10-12 23:31:20,629][44958] Updated weights for policy 0, policy_version 82740 (0.0007) [2023-10-12 23:31:20,996][44958] Updated weights for policy 0, policy_version 82750 (0.0009) [2023-10-12 23:31:21,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 169869312. Throughput: 0: 1637.9, 1: 1652.3. Samples: 42471260. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:31:21,444][43579] Avg episode reward: [(0, '267.330'), (1, '277.500')] [2023-10-12 23:31:21,816][44959] Updated weights for policy 1, policy_version 83140 (0.0007) [2023-10-12 23:31:22,186][44959] Updated weights for policy 1, policy_version 83150 (0.0007) [2023-10-12 23:31:22,550][44959] Updated weights for policy 1, policy_version 83160 (0.0008) [2023-10-12 23:31:25,078][44958] Updated weights for policy 0, policy_version 82760 (0.0009) [2023-10-12 23:31:25,450][44958] Updated weights for policy 0, policy_version 82770 (0.0008) [2023-10-12 23:31:25,825][44958] Updated weights for policy 0, policy_version 82780 (0.0010) [2023-10-12 23:31:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 169934848. Throughput: 0: 1644.5, 1: 1660.1. Samples: 42491466. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:31:26,443][43579] Avg episode reward: [(0, '261.950'), (1, '279.520')] [2023-10-12 23:31:26,639][44959] Updated weights for policy 1, policy_version 83170 (0.0010) [2023-10-12 23:31:27,008][44959] Updated weights for policy 1, policy_version 83180 (0.0007) [2023-10-12 23:31:27,376][44959] Updated weights for policy 1, policy_version 83190 (0.0007) [2023-10-12 23:31:27,744][44959] Updated weights for policy 1, policy_version 83200 (0.0009) [2023-10-12 23:31:30,003][44958] Updated weights for policy 0, policy_version 82790 (0.0009) [2023-10-12 23:31:30,388][44958] Updated weights for policy 0, policy_version 82800 (0.0009) [2023-10-12 23:31:30,761][44958] Updated weights for policy 0, policy_version 82810 (0.0010) [2023-10-12 23:31:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 170000384. Throughput: 0: 1644.8, 1: 1653.5. Samples: 42510722. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:31:31,443][43579] Avg episode reward: [(0, '267.780'), (1, '279.900')] [2023-10-12 23:31:31,795][44959] Updated weights for policy 1, policy_version 83210 (0.0008) [2023-10-12 23:31:32,172][44959] Updated weights for policy 1, policy_version 83220 (0.0009) [2023-10-12 23:31:32,542][44959] Updated weights for policy 1, policy_version 83230 (0.0009) [2023-10-12 23:31:35,119][44958] Updated weights for policy 0, policy_version 82820 (0.0009) [2023-10-12 23:31:35,516][44958] Updated weights for policy 0, policy_version 82830 (0.0009) [2023-10-12 23:31:35,883][44958] Updated weights for policy 0, policy_version 82840 (0.0010) [2023-10-12 23:31:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170065920. Throughput: 0: 1642.9, 1: 1652.4. Samples: 42520844. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:31:36,443][43579] Avg episode reward: [(0, '261.890'), (1, '282.990')] [2023-10-12 23:31:36,885][44959] Updated weights for policy 1, policy_version 83240 (0.0009) [2023-10-12 23:31:37,261][44959] Updated weights for policy 1, policy_version 83250 (0.0008) [2023-10-12 23:31:37,633][44959] Updated weights for policy 1, policy_version 83260 (0.0010) [2023-10-12 23:31:40,025][44958] Updated weights for policy 0, policy_version 82850 (0.0007) [2023-10-12 23:31:40,407][44958] Updated weights for policy 0, policy_version 82860 (0.0009) [2023-10-12 23:31:40,781][44958] Updated weights for policy 0, policy_version 82870 (0.0010) [2023-10-12 23:31:41,157][44958] Updated weights for policy 0, policy_version 82880 (0.0009) [2023-10-12 23:31:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170131456. Throughput: 0: 1645.9, 1: 1656.1. Samples: 42540676. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:31:41,444][43579] Avg episode reward: [(0, '258.670'), (1, '283.940')] [2023-10-12 23:31:41,666][44959] Updated weights for policy 1, policy_version 83270 (0.0009) [2023-10-12 23:31:42,048][44959] Updated weights for policy 1, policy_version 83280 (0.0009) [2023-10-12 23:31:42,414][44959] Updated weights for policy 1, policy_version 83290 (0.0008) [2023-10-12 23:31:45,495][44958] Updated weights for policy 0, policy_version 82890 (0.0007) [2023-10-12 23:31:45,859][44958] Updated weights for policy 0, policy_version 82900 (0.0008) [2023-10-12 23:31:46,229][44958] Updated weights for policy 0, policy_version 82910 (0.0010) [2023-10-12 23:31:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 170196992. Throughput: 0: 1641.9, 1: 1652.6. Samples: 42559868. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:31:46,444][43579] Avg episode reward: [(0, '263.740'), (1, '284.550')] [2023-10-12 23:31:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000082912_84901888.pth... [2023-10-12 23:31:46,488][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000081376_83329024.pth [2023-10-12 23:31:46,513][44959] Updated weights for policy 1, policy_version 83300 (0.0009) [2023-10-12 23:31:46,875][44959] Updated weights for policy 1, policy_version 83310 (0.0009) [2023-10-12 23:31:47,242][44959] Updated weights for policy 1, policy_version 83320 (0.0008) [2023-10-12 23:31:47,535][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000083328_85327872.pth... [2023-10-12 23:31:47,576][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000081760_83722240.pth [2023-10-12 23:31:50,343][44958] Updated weights for policy 0, policy_version 82920 (0.0008) [2023-10-12 23:31:50,714][44958] Updated weights for policy 0, policy_version 82930 (0.0008) [2023-10-12 23:31:51,092][44958] Updated weights for policy 0, policy_version 82940 (0.0008) [2023-10-12 23:31:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170262528. Throughput: 0: 1633.5, 1: 1648.1. Samples: 42569526. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:31:51,443][43579] Avg episode reward: [(0, '272.260'), (1, '289.510')] [2023-10-12 23:31:51,444][44959] Updated weights for policy 1, policy_version 83330 (0.0009) [2023-10-12 23:31:51,816][44959] Updated weights for policy 1, policy_version 83340 (0.0009) [2023-10-12 23:31:52,185][44959] Updated weights for policy 1, policy_version 83350 (0.0009) [2023-10-12 23:31:52,558][44959] Updated weights for policy 1, policy_version 83360 (0.0008) [2023-10-12 23:31:55,395][44958] Updated weights for policy 0, policy_version 82950 (0.0009) [2023-10-12 23:31:55,764][44958] Updated weights for policy 0, policy_version 82960 (0.0007) [2023-10-12 23:31:56,130][44958] Updated weights for policy 0, policy_version 82970 (0.0007) [2023-10-12 23:31:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170328064. Throughput: 0: 1644.2, 1: 1649.4. Samples: 42589926. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:31:56,443][43579] Avg episode reward: [(0, '277.610'), (1, '283.030')] [2023-10-12 23:31:56,784][44959] Updated weights for policy 1, policy_version 83370 (0.0008) [2023-10-12 23:31:57,152][44959] Updated weights for policy 1, policy_version 83380 (0.0008) [2023-10-12 23:31:57,530][44959] Updated weights for policy 1, policy_version 83390 (0.0010) [2023-10-12 23:32:00,270][44958] Updated weights for policy 0, policy_version 82980 (0.0009) [2023-10-12 23:32:00,639][44958] Updated weights for policy 0, policy_version 82990 (0.0009) [2023-10-12 23:32:01,010][44958] Updated weights for policy 0, policy_version 83000 (0.0008) [2023-10-12 23:32:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170393600. Throughput: 0: 1644.4, 1: 1654.4. Samples: 42609546. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:32:01,443][43579] Avg episode reward: [(0, '269.600'), (1, '284.000')] [2023-10-12 23:32:01,575][44959] Updated weights for policy 1, policy_version 83400 (0.0009) [2023-10-12 23:32:01,954][44959] Updated weights for policy 1, policy_version 83410 (0.0009) [2023-10-12 23:32:02,314][44959] Updated weights for policy 1, policy_version 83420 (0.0008) [2023-10-12 23:32:04,767][44958] Updated weights for policy 0, policy_version 83010 (0.0009) [2023-10-12 23:32:05,139][44958] Updated weights for policy 0, policy_version 83020 (0.0009) [2023-10-12 23:32:05,504][44958] Updated weights for policy 0, policy_version 83030 (0.0011) [2023-10-12 23:32:05,869][44958] Updated weights for policy 0, policy_version 83040 (0.0007) [2023-10-12 23:32:06,429][44959] Updated weights for policy 1, policy_version 83430 (0.0008) [2023-10-12 23:32:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 170459136. Throughput: 0: 1646.8, 1: 1656.5. Samples: 42619904. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:32:06,443][43579] Avg episode reward: [(0, '274.730'), (1, '286.650')] [2023-10-12 23:32:06,799][44959] Updated weights for policy 1, policy_version 83440 (0.0008) [2023-10-12 23:32:07,175][44959] Updated weights for policy 1, policy_version 83450 (0.0007) [2023-10-12 23:32:09,918][44958] Updated weights for policy 0, policy_version 83050 (0.0009) [2023-10-12 23:32:10,286][44958] Updated weights for policy 0, policy_version 83060 (0.0007) [2023-10-12 23:32:10,669][44958] Updated weights for policy 0, policy_version 83070 (0.0008) [2023-10-12 23:32:11,440][44959] Updated weights for policy 1, policy_version 83460 (0.0008) [2023-10-12 23:32:11,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170524672. Throughput: 0: 1639.7, 1: 1647.2. Samples: 42639380. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:32:11,443][43579] Avg episode reward: [(0, '271.230'), (1, '288.060')] [2023-10-12 23:32:11,816][44959] Updated weights for policy 1, policy_version 83470 (0.0010) [2023-10-12 23:32:12,189][44959] Updated weights for policy 1, policy_version 83480 (0.0009) [2023-10-12 23:32:14,862][44958] Updated weights for policy 0, policy_version 83080 (0.0007) [2023-10-12 23:32:15,233][44958] Updated weights for policy 0, policy_version 83090 (0.0008) [2023-10-12 23:32:15,602][44958] Updated weights for policy 0, policy_version 83100 (0.0009) [2023-10-12 23:32:16,046][44959] Updated weights for policy 1, policy_version 83490 (0.0008) [2023-10-12 23:32:16,411][44959] Updated weights for policy 1, policy_version 83500 (0.0007) [2023-10-12 23:32:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170590208. Throughput: 0: 1646.6, 1: 1654.5. Samples: 42659272. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:32:16,443][43579] Avg episode reward: [(0, '265.340'), (1, '288.180')] [2023-10-12 23:32:16,780][44959] Updated weights for policy 1, policy_version 83510 (0.0009) [2023-10-12 23:32:17,157][44959] Updated weights for policy 1, policy_version 83520 (0.0007) [2023-10-12 23:32:19,722][44958] Updated weights for policy 0, policy_version 83110 (0.0009) [2023-10-12 23:32:20,107][44958] Updated weights for policy 0, policy_version 83120 (0.0009) [2023-10-12 23:32:20,476][44958] Updated weights for policy 0, policy_version 83130 (0.0008) [2023-10-12 23:32:21,349][44959] Updated weights for policy 1, policy_version 83530 (0.0008) [2023-10-12 23:32:21,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170655744. Throughput: 0: 1645.6, 1: 1655.9. Samples: 42669410. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:32:21,444][43579] Avg episode reward: [(0, '263.150'), (1, '288.320')] [2023-10-12 23:32:21,732][44959] Updated weights for policy 1, policy_version 83540 (0.0009) [2023-10-12 23:32:22,101][44959] Updated weights for policy 1, policy_version 83550 (0.0010) [2023-10-12 23:32:24,708][44958] Updated weights for policy 0, policy_version 83140 (0.0008) [2023-10-12 23:32:25,101][44958] Updated weights for policy 0, policy_version 83150 (0.0007) [2023-10-12 23:32:25,474][44958] Updated weights for policy 0, policy_version 83160 (0.0008) [2023-10-12 23:32:26,358][44959] Updated weights for policy 1, policy_version 83560 (0.0009) [2023-10-12 23:32:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170721280. Throughput: 0: 1640.8, 1: 1651.3. Samples: 42688820. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:32:26,443][43579] Avg episode reward: [(0, '271.260'), (1, '292.410')] [2023-10-12 23:32:26,733][44959] Updated weights for policy 1, policy_version 83570 (0.0007) [2023-10-12 23:32:27,097][44959] Updated weights for policy 1, policy_version 83580 (0.0009) [2023-10-12 23:32:29,572][44958] Updated weights for policy 0, policy_version 83170 (0.0008) [2023-10-12 23:32:29,934][44958] Updated weights for policy 0, policy_version 83180 (0.0008) [2023-10-12 23:32:30,321][44958] Updated weights for policy 0, policy_version 83190 (0.0009) [2023-10-12 23:32:30,680][44958] Updated weights for policy 0, policy_version 83200 (0.0011) [2023-10-12 23:32:31,401][44959] Updated weights for policy 1, policy_version 83590 (0.0008) [2023-10-12 23:32:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170786816. Throughput: 0: 1649.1, 1: 1658.4. Samples: 42708702. Policy #0 lag: (min: 4.0, avg: 18.4, max: 36.0) [2023-10-12 23:32:31,443][43579] Avg episode reward: [(0, '264.520'), (1, '288.080')] [2023-10-12 23:32:31,779][44959] Updated weights for policy 1, policy_version 83600 (0.0009) [2023-10-12 23:32:32,155][44959] Updated weights for policy 1, policy_version 83610 (0.0011) [2023-10-12 23:32:35,066][44958] Updated weights for policy 0, policy_version 83210 (0.0008) [2023-10-12 23:32:35,451][44958] Updated weights for policy 0, policy_version 83220 (0.0009) [2023-10-12 23:32:35,816][44958] Updated weights for policy 0, policy_version 83230 (0.0010) [2023-10-12 23:32:36,322][44959] Updated weights for policy 1, policy_version 83620 (0.0009) [2023-10-12 23:32:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170852352. Throughput: 0: 1659.4, 1: 1655.6. Samples: 42718702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:32:36,444][43579] Avg episode reward: [(0, '265.030'), (1, '288.720')] [2023-10-12 23:32:36,693][44959] Updated weights for policy 1, policy_version 83630 (0.0009) [2023-10-12 23:32:37,062][44959] Updated weights for policy 1, policy_version 83640 (0.0010) [2023-10-12 23:32:39,964][44958] Updated weights for policy 0, policy_version 83240 (0.0009) [2023-10-12 23:32:40,339][44958] Updated weights for policy 0, policy_version 83250 (0.0009) [2023-10-12 23:32:40,703][44958] Updated weights for policy 0, policy_version 83260 (0.0009) [2023-10-12 23:32:41,123][44959] Updated weights for policy 1, policy_version 83650 (0.0009) [2023-10-12 23:32:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 170917888. Throughput: 0: 1639.7, 1: 1656.6. Samples: 42738260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:32:41,443][43579] Avg episode reward: [(0, '269.590'), (1, '282.840')] [2023-10-12 23:32:41,488][44959] Updated weights for policy 1, policy_version 83660 (0.0007) [2023-10-12 23:32:41,859][44959] Updated weights for policy 1, policy_version 83670 (0.0007) [2023-10-12 23:32:42,226][44959] Updated weights for policy 1, policy_version 83680 (0.0007) [2023-10-12 23:32:44,792][44958] Updated weights for policy 0, policy_version 83270 (0.0008) [2023-10-12 23:32:45,159][44958] Updated weights for policy 0, policy_version 83280 (0.0009) [2023-10-12 23:32:45,530][44958] Updated weights for policy 0, policy_version 83290 (0.0008) [2023-10-12 23:32:46,404][44959] Updated weights for policy 1, policy_version 83690 (0.0007) [2023-10-12 23:32:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 170983424. Throughput: 0: 1644.7, 1: 1656.3. Samples: 42758092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:32:46,443][43579] Avg episode reward: [(0, '275.890'), (1, '282.210')] [2023-10-12 23:32:46,777][44959] Updated weights for policy 1, policy_version 83700 (0.0009) [2023-10-12 23:32:47,139][44959] Updated weights for policy 1, policy_version 83710 (0.0007) [2023-10-12 23:32:49,793][44958] Updated weights for policy 0, policy_version 83300 (0.0008) [2023-10-12 23:32:50,169][44958] Updated weights for policy 0, policy_version 83310 (0.0008) [2023-10-12 23:32:50,529][44958] Updated weights for policy 0, policy_version 83320 (0.0007) [2023-10-12 23:32:51,425][44959] Updated weights for policy 1, policy_version 83720 (0.0008) [2023-10-12 23:32:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171048960. Throughput: 0: 1640.9, 1: 1650.8. Samples: 42768028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:32:51,443][43579] Avg episode reward: [(0, '273.030'), (1, '281.150')] [2023-10-12 23:32:51,795][44959] Updated weights for policy 1, policy_version 83730 (0.0009) [2023-10-12 23:32:52,165][44959] Updated weights for policy 1, policy_version 83740 (0.0009) [2023-10-12 23:32:54,681][44958] Updated weights for policy 0, policy_version 83330 (0.0009) [2023-10-12 23:32:55,055][44958] Updated weights for policy 0, policy_version 83340 (0.0009) [2023-10-12 23:32:55,436][44958] Updated weights for policy 0, policy_version 83350 (0.0007) [2023-10-12 23:32:55,803][44958] Updated weights for policy 0, policy_version 83360 (0.0007) [2023-10-12 23:32:56,284][44959] Updated weights for policy 1, policy_version 83750 (0.0009) [2023-10-12 23:32:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171114496. Throughput: 0: 1644.7, 1: 1651.2. Samples: 42787692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:32:56,443][43579] Avg episode reward: [(0, '273.120'), (1, '282.510')] [2023-10-12 23:32:56,647][44959] Updated weights for policy 1, policy_version 83760 (0.0009) [2023-10-12 23:32:57,008][44959] Updated weights for policy 1, policy_version 83770 (0.0009) [2023-10-12 23:32:59,908][44958] Updated weights for policy 0, policy_version 83370 (0.0009) [2023-10-12 23:33:00,285][44958] Updated weights for policy 0, policy_version 83380 (0.0009) [2023-10-12 23:33:00,652][44958] Updated weights for policy 0, policy_version 83390 (0.0008) [2023-10-12 23:33:01,297][44959] Updated weights for policy 1, policy_version 83780 (0.0008) [2023-10-12 23:33:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171180032. Throughput: 0: 1648.1, 1: 1644.0. Samples: 42807418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:33:01,444][43579] Avg episode reward: [(0, '279.750'), (1, '285.500')] [2023-10-12 23:33:01,671][44959] Updated weights for policy 1, policy_version 83790 (0.0010) [2023-10-12 23:33:02,044][44959] Updated weights for policy 1, policy_version 83800 (0.0008) [2023-10-12 23:33:04,819][44958] Updated weights for policy 0, policy_version 83400 (0.0008) [2023-10-12 23:33:05,188][44958] Updated weights for policy 0, policy_version 83410 (0.0009) [2023-10-12 23:33:05,561][44958] Updated weights for policy 0, policy_version 83420 (0.0007) [2023-10-12 23:33:06,067][44959] Updated weights for policy 1, policy_version 83810 (0.0008) [2023-10-12 23:33:06,431][44959] Updated weights for policy 1, policy_version 83820 (0.0009) [2023-10-12 23:33:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171245568. Throughput: 0: 1645.9, 1: 1641.6. Samples: 42817348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:33:06,443][43579] Avg episode reward: [(0, '280.800'), (1, '284.360')] [2023-10-12 23:33:06,801][44959] Updated weights for policy 1, policy_version 83830 (0.0010) [2023-10-12 23:33:07,180][44959] Updated weights for policy 1, policy_version 83840 (0.0009) [2023-10-12 23:33:09,626][44958] Updated weights for policy 0, policy_version 83430 (0.0008) [2023-10-12 23:33:10,005][44958] Updated weights for policy 0, policy_version 83440 (0.0007) [2023-10-12 23:33:10,374][44958] Updated weights for policy 0, policy_version 83450 (0.0009) [2023-10-12 23:33:11,139][44959] Updated weights for policy 1, policy_version 83850 (0.0011) [2023-10-12 23:33:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171311104. Throughput: 0: 1645.3, 1: 1649.8. Samples: 42837100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:33:11,443][43579] Avg episode reward: [(0, '279.140'), (1, '282.240')] [2023-10-12 23:33:11,516][44959] Updated weights for policy 1, policy_version 83860 (0.0010) [2023-10-12 23:33:11,871][44959] Updated weights for policy 1, policy_version 83870 (0.0010) [2023-10-12 23:33:14,720][44958] Updated weights for policy 0, policy_version 83460 (0.0009) [2023-10-12 23:33:15,112][44958] Updated weights for policy 0, policy_version 83470 (0.0010) [2023-10-12 23:33:15,484][44958] Updated weights for policy 0, policy_version 83480 (0.0008) [2023-10-12 23:33:16,066][44959] Updated weights for policy 1, policy_version 83880 (0.0009) [2023-10-12 23:33:16,436][44959] Updated weights for policy 1, policy_version 83890 (0.0008) [2023-10-12 23:33:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171376640. Throughput: 0: 1643.2, 1: 1638.6. Samples: 42856384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:33:16,444][43579] Avg episode reward: [(0, '276.080'), (1, '282.510')] [2023-10-12 23:33:16,810][44959] Updated weights for policy 1, policy_version 83900 (0.0008) [2023-10-12 23:33:19,760][44958] Updated weights for policy 0, policy_version 83490 (0.0009) [2023-10-12 23:33:20,130][44958] Updated weights for policy 0, policy_version 83500 (0.0010) [2023-10-12 23:33:20,514][44958] Updated weights for policy 0, policy_version 83510 (0.0010) [2023-10-12 23:33:20,886][44958] Updated weights for policy 0, policy_version 83520 (0.0011) [2023-10-12 23:33:21,028][44959] Updated weights for policy 1, policy_version 83910 (0.0008) [2023-10-12 23:33:21,397][44959] Updated weights for policy 1, policy_version 83920 (0.0010) [2023-10-12 23:33:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171442176. Throughput: 0: 1638.8, 1: 1649.5. Samples: 42866672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:33:21,443][43579] Avg episode reward: [(0, '279.370'), (1, '281.290')] [2023-10-12 23:33:21,770][44959] Updated weights for policy 1, policy_version 83930 (0.0008) [2023-10-12 23:33:25,059][44958] Updated weights for policy 0, policy_version 83530 (0.0007) [2023-10-12 23:33:25,426][44958] Updated weights for policy 0, policy_version 83540 (0.0008) [2023-10-12 23:33:25,793][44958] Updated weights for policy 0, policy_version 83550 (0.0009) [2023-10-12 23:33:26,033][44959] Updated weights for policy 1, policy_version 83940 (0.0007) [2023-10-12 23:33:26,404][44959] Updated weights for policy 1, policy_version 83950 (0.0007) [2023-10-12 23:33:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171507712. Throughput: 0: 1642.8, 1: 1647.1. Samples: 42886306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:33:26,443][43579] Avg episode reward: [(0, '282.120'), (1, '280.300')] [2023-10-12 23:33:26,769][44959] Updated weights for policy 1, policy_version 83960 (0.0008) [2023-10-12 23:33:29,829][44958] Updated weights for policy 0, policy_version 83560 (0.0010) [2023-10-12 23:33:30,203][44958] Updated weights for policy 0, policy_version 83570 (0.0009) [2023-10-12 23:33:30,572][44958] Updated weights for policy 0, policy_version 83580 (0.0009) [2023-10-12 23:33:30,924][44959] Updated weights for policy 1, policy_version 83970 (0.0008) [2023-10-12 23:33:31,296][44959] Updated weights for policy 1, policy_version 83980 (0.0010) [2023-10-12 23:33:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171573248. Throughput: 0: 1642.7, 1: 1640.8. Samples: 42905850. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:33:31,443][43579] Avg episode reward: [(0, '281.200'), (1, '280.890')] [2023-10-12 23:33:31,669][44959] Updated weights for policy 1, policy_version 83990 (0.0008) [2023-10-12 23:33:32,038][44959] Updated weights for policy 1, policy_version 84000 (0.0008) [2023-10-12 23:33:34,719][44958] Updated weights for policy 0, policy_version 83590 (0.0010) [2023-10-12 23:33:35,103][44958] Updated weights for policy 0, policy_version 83600 (0.0010) [2023-10-12 23:33:35,463][44958] Updated weights for policy 0, policy_version 83610 (0.0010) [2023-10-12 23:33:36,238][44959] Updated weights for policy 1, policy_version 84010 (0.0011) [2023-10-12 23:33:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171638784. Throughput: 0: 1643.7, 1: 1643.4. Samples: 42915946. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:33:36,443][43579] Avg episode reward: [(0, '280.720'), (1, '282.450')] [2023-10-12 23:33:36,608][44959] Updated weights for policy 1, policy_version 84020 (0.0010) [2023-10-12 23:33:36,975][44959] Updated weights for policy 1, policy_version 84030 (0.0008) [2023-10-12 23:33:39,652][44958] Updated weights for policy 0, policy_version 83620 (0.0010) [2023-10-12 23:33:40,007][44958] Updated weights for policy 0, policy_version 83630 (0.0009) [2023-10-12 23:33:40,392][44958] Updated weights for policy 0, policy_version 83640 (0.0009) [2023-10-12 23:33:40,973][44959] Updated weights for policy 1, policy_version 84040 (0.0008) [2023-10-12 23:33:41,344][44959] Updated weights for policy 1, policy_version 84050 (0.0007) [2023-10-12 23:33:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171704320. Throughput: 0: 1638.6, 1: 1650.5. Samples: 42935702. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:33:41,443][43579] Avg episode reward: [(0, '281.410'), (1, '282.770')] [2023-10-12 23:33:41,713][44959] Updated weights for policy 1, policy_version 84060 (0.0009) [2023-10-12 23:33:44,536][44958] Updated weights for policy 0, policy_version 83650 (0.0008) [2023-10-12 23:33:44,911][44958] Updated weights for policy 0, policy_version 83660 (0.0007) [2023-10-12 23:33:45,281][44958] Updated weights for policy 0, policy_version 83670 (0.0007) [2023-10-12 23:33:45,646][44958] Updated weights for policy 0, policy_version 83680 (0.0009) [2023-10-12 23:33:45,718][44959] Updated weights for policy 1, policy_version 84070 (0.0007) [2023-10-12 23:33:46,075][44959] Updated weights for policy 1, policy_version 84080 (0.0007) [2023-10-12 23:33:46,443][43579] Fps is (10 sec: 13106.4, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 171769856. Throughput: 0: 1632.3, 1: 1639.2. Samples: 42954636. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:33:46,444][43579] Avg episode reward: [(0, '286.660'), (1, '280.650')] [2023-10-12 23:33:46,450][44959] Updated weights for policy 1, policy_version 84090 (0.0008) [2023-10-12 23:33:46,455][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000083680_85688320.pth... [2023-10-12 23:33:46,487][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000082144_84115456.pth [2023-10-12 23:33:46,662][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000084096_86114304.pth... [2023-10-12 23:33:46,691][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000082528_84508672.pth [2023-10-12 23:33:50,077][44958] Updated weights for policy 0, policy_version 83690 (0.0010) [2023-10-12 23:33:50,446][44958] Updated weights for policy 0, policy_version 83700 (0.0008) [2023-10-12 23:33:50,657][44959] Updated weights for policy 1, policy_version 84100 (0.0008) [2023-10-12 23:33:50,824][44958] Updated weights for policy 0, policy_version 83710 (0.0010) [2023-10-12 23:33:51,029][44959] Updated weights for policy 1, policy_version 84110 (0.0008) [2023-10-12 23:33:51,408][44959] Updated weights for policy 1, policy_version 84120 (0.0009) [2023-10-12 23:33:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171835392. Throughput: 0: 1633.1, 1: 1652.0. Samples: 42965176. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:33:51,443][43579] Avg episode reward: [(0, '281.680'), (1, '281.810')] [2023-10-12 23:33:54,984][44958] Updated weights for policy 0, policy_version 83720 (0.0008) [2023-10-12 23:33:55,359][44958] Updated weights for policy 0, policy_version 83730 (0.0009) [2023-10-12 23:33:55,639][44959] Updated weights for policy 1, policy_version 84130 (0.0010) [2023-10-12 23:33:55,722][44958] Updated weights for policy 0, policy_version 83740 (0.0009) [2023-10-12 23:33:56,011][44959] Updated weights for policy 1, policy_version 84140 (0.0008) [2023-10-12 23:33:56,382][44959] Updated weights for policy 1, policy_version 84150 (0.0010) [2023-10-12 23:33:56,442][43579] Fps is (10 sec: 13107.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 171900928. Throughput: 0: 1640.1, 1: 1647.8. Samples: 42985054. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:33:56,443][43579] Avg episode reward: [(0, '282.310'), (1, '282.430')] [2023-10-12 23:33:56,748][44959] Updated weights for policy 1, policy_version 84160 (0.0008) [2023-10-12 23:33:59,895][44958] Updated weights for policy 0, policy_version 83750 (0.0008) [2023-10-12 23:34:00,278][44958] Updated weights for policy 0, policy_version 83760 (0.0009) [2023-10-12 23:34:00,648][44958] Updated weights for policy 0, policy_version 83770 (0.0009) [2023-10-12 23:34:01,069][44959] Updated weights for policy 1, policy_version 84170 (0.0008) [2023-10-12 23:34:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 171966464. Throughput: 0: 1635.7, 1: 1641.7. Samples: 43003866. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:34:01,443][43579] Avg episode reward: [(0, '277.830'), (1, '284.130')] [2023-10-12 23:34:01,445][44959] Updated weights for policy 1, policy_version 84180 (0.0009) [2023-10-12 23:34:01,818][44959] Updated weights for policy 1, policy_version 84190 (0.0009) [2023-10-12 23:34:04,630][44958] Updated weights for policy 0, policy_version 83780 (0.0008) [2023-10-12 23:34:05,008][44958] Updated weights for policy 0, policy_version 83790 (0.0008) [2023-10-12 23:34:05,371][44958] Updated weights for policy 0, policy_version 83800 (0.0009) [2023-10-12 23:34:05,926][44959] Updated weights for policy 1, policy_version 84200 (0.0008) [2023-10-12 23:34:06,293][44959] Updated weights for policy 1, policy_version 84210 (0.0008) [2023-10-12 23:34:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172032000. Throughput: 0: 1639.8, 1: 1645.0. Samples: 43014488. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:34:06,443][43579] Avg episode reward: [(0, '278.200'), (1, '284.890')] [2023-10-12 23:34:06,656][44959] Updated weights for policy 1, policy_version 84220 (0.0009) [2023-10-12 23:34:09,482][44958] Updated weights for policy 0, policy_version 83810 (0.0008) [2023-10-12 23:34:09,851][44958] Updated weights for policy 0, policy_version 83820 (0.0007) [2023-10-12 23:34:10,227][44958] Updated weights for policy 0, policy_version 83830 (0.0007) [2023-10-12 23:34:10,607][44958] Updated weights for policy 0, policy_version 83840 (0.0008) [2023-10-12 23:34:10,772][44959] Updated weights for policy 1, policy_version 84230 (0.0007) [2023-10-12 23:34:11,146][44959] Updated weights for policy 1, policy_version 84240 (0.0007) [2023-10-12 23:34:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172097536. Throughput: 0: 1635.2, 1: 1646.3. Samples: 43033974. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:34:11,443][43579] Avg episode reward: [(0, '279.330'), (1, '288.890')] [2023-10-12 23:34:11,511][44959] Updated weights for policy 1, policy_version 84250 (0.0009) [2023-10-12 23:34:14,795][44958] Updated weights for policy 0, policy_version 83850 (0.0007) [2023-10-12 23:34:15,169][44958] Updated weights for policy 0, policy_version 83860 (0.0007) [2023-10-12 23:34:15,540][44958] Updated weights for policy 0, policy_version 83870 (0.0009) [2023-10-12 23:34:15,902][44959] Updated weights for policy 1, policy_version 84260 (0.0009) [2023-10-12 23:34:16,274][44959] Updated weights for policy 1, policy_version 84270 (0.0007) [2023-10-12 23:34:16,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172163072. Throughput: 0: 1643.8, 1: 1636.3. Samples: 43053454. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:34:16,443][43579] Avg episode reward: [(0, '282.770'), (1, '288.290')] [2023-10-12 23:34:16,653][44959] Updated weights for policy 1, policy_version 84280 (0.0007) [2023-10-12 23:34:19,698][44958] Updated weights for policy 0, policy_version 83880 (0.0007) [2023-10-12 23:34:20,072][44958] Updated weights for policy 0, policy_version 83890 (0.0007) [2023-10-12 23:34:20,435][44958] Updated weights for policy 0, policy_version 83900 (0.0007) [2023-10-12 23:34:20,762][44959] Updated weights for policy 1, policy_version 84290 (0.0008) [2023-10-12 23:34:21,134][44959] Updated weights for policy 1, policy_version 84300 (0.0010) [2023-10-12 23:34:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172228608. Throughput: 0: 1643.4, 1: 1640.9. Samples: 43063742. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-12 23:34:21,444][43579] Avg episode reward: [(0, '278.060'), (1, '287.680')] [2023-10-12 23:34:21,516][44959] Updated weights for policy 1, policy_version 84310 (0.0009) [2023-10-12 23:34:21,873][44959] Updated weights for policy 1, policy_version 84320 (0.0009) [2023-10-12 23:34:24,518][44958] Updated weights for policy 0, policy_version 83910 (0.0010) [2023-10-12 23:34:24,897][44958] Updated weights for policy 0, policy_version 83920 (0.0008) [2023-10-12 23:34:25,268][44958] Updated weights for policy 0, policy_version 83930 (0.0009) [2023-10-12 23:34:26,078][44959] Updated weights for policy 1, policy_version 84330 (0.0009) [2023-10-12 23:34:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172294144. Throughput: 0: 1639.9, 1: 1639.0. Samples: 43083254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:34:26,444][43579] Avg episode reward: [(0, '271.690'), (1, '285.010')] [2023-10-12 23:34:26,449][44959] Updated weights for policy 1, policy_version 84340 (0.0008) [2023-10-12 23:34:26,818][44959] Updated weights for policy 1, policy_version 84350 (0.0008) [2023-10-12 23:34:29,485][44958] Updated weights for policy 0, policy_version 83940 (0.0009) [2023-10-12 23:34:29,853][44958] Updated weights for policy 0, policy_version 83950 (0.0009) [2023-10-12 23:34:30,229][44958] Updated weights for policy 0, policy_version 83960 (0.0008) [2023-10-12 23:34:30,742][44959] Updated weights for policy 1, policy_version 84360 (0.0009) [2023-10-12 23:34:31,101][44959] Updated weights for policy 1, policy_version 84370 (0.0008) [2023-10-12 23:34:31,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 172359680. Throughput: 0: 1647.2, 1: 1643.8. Samples: 43102734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:34:31,444][43579] Avg episode reward: [(0, '273.180'), (1, '287.600')] [2023-10-12 23:34:31,473][44959] Updated weights for policy 1, policy_version 84380 (0.0011) [2023-10-12 23:34:34,451][44958] Updated weights for policy 0, policy_version 83970 (0.0009) [2023-10-12 23:34:34,829][44958] Updated weights for policy 0, policy_version 83980 (0.0007) [2023-10-12 23:34:35,206][44958] Updated weights for policy 0, policy_version 83990 (0.0009) [2023-10-12 23:34:35,579][44958] Updated weights for policy 0, policy_version 84000 (0.0009) [2023-10-12 23:34:35,646][44959] Updated weights for policy 1, policy_version 84390 (0.0010) [2023-10-12 23:34:36,016][44959] Updated weights for policy 1, policy_version 84400 (0.0009) [2023-10-12 23:34:36,379][44959] Updated weights for policy 1, policy_version 84410 (0.0010) [2023-10-12 23:34:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172425216. Throughput: 0: 1647.9, 1: 1646.9. Samples: 43113442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:34:36,443][43579] Avg episode reward: [(0, '270.380'), (1, '286.810')] [2023-10-12 23:34:39,908][44958] Updated weights for policy 0, policy_version 84010 (0.0009) [2023-10-12 23:34:40,283][44958] Updated weights for policy 0, policy_version 84020 (0.0007) [2023-10-12 23:34:40,665][44958] Updated weights for policy 0, policy_version 84030 (0.0007) [2023-10-12 23:34:40,875][44959] Updated weights for policy 1, policy_version 84420 (0.0011) [2023-10-12 23:34:41,245][44959] Updated weights for policy 1, policy_version 84430 (0.0010) [2023-10-12 23:34:41,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 172490752. Throughput: 0: 1643.9, 1: 1642.3. Samples: 43132930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:34:41,444][43579] Avg episode reward: [(0, '269.440'), (1, '285.900')] [2023-10-12 23:34:41,608][44959] Updated weights for policy 1, policy_version 84440 (0.0010) [2023-10-12 23:34:44,869][44958] Updated weights for policy 0, policy_version 84040 (0.0007) [2023-10-12 23:34:45,248][44958] Updated weights for policy 0, policy_version 84050 (0.0010) [2023-10-12 23:34:45,622][44958] Updated weights for policy 0, policy_version 84060 (0.0011) [2023-10-12 23:34:45,858][44959] Updated weights for policy 1, policy_version 84450 (0.0008) [2023-10-12 23:34:46,269][44959] Updated weights for policy 1, policy_version 84460 (0.0008) [2023-10-12 23:34:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 172556288. Throughput: 0: 1645.8, 1: 1644.1. Samples: 43151910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:34:46,443][43579] Avg episode reward: [(0, '270.450'), (1, '287.030')] [2023-10-12 23:34:46,636][44959] Updated weights for policy 1, policy_version 84470 (0.0007) [2023-10-12 23:34:47,003][44959] Updated weights for policy 1, policy_version 84480 (0.0009) [2023-10-12 23:34:49,675][44958] Updated weights for policy 0, policy_version 84070 (0.0008) [2023-10-12 23:34:50,049][44958] Updated weights for policy 0, policy_version 84080 (0.0009) [2023-10-12 23:34:50,424][44958] Updated weights for policy 0, policy_version 84090 (0.0008) [2023-10-12 23:34:51,145][44959] Updated weights for policy 1, policy_version 84490 (0.0009) [2023-10-12 23:34:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172621824. Throughput: 0: 1644.9, 1: 1638.1. Samples: 43162226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:34:51,444][43579] Avg episode reward: [(0, '274.570'), (1, '288.010')] [2023-10-12 23:34:51,503][44959] Updated weights for policy 1, policy_version 84500 (0.0007) [2023-10-12 23:34:51,874][44959] Updated weights for policy 1, policy_version 84510 (0.0008) [2023-10-12 23:34:54,458][44958] Updated weights for policy 0, policy_version 84100 (0.0009) [2023-10-12 23:34:54,833][44958] Updated weights for policy 0, policy_version 84110 (0.0009) [2023-10-12 23:34:55,211][44958] Updated weights for policy 0, policy_version 84120 (0.0010) [2023-10-12 23:34:55,952][44959] Updated weights for policy 1, policy_version 84520 (0.0007) [2023-10-12 23:34:56,315][44959] Updated weights for policy 1, policy_version 84530 (0.0008) [2023-10-12 23:34:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172687360. Throughput: 0: 1644.7, 1: 1640.0. Samples: 43181784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:34:56,444][43579] Avg episode reward: [(0, '280.230'), (1, '289.250')] [2023-10-12 23:34:56,692][44959] Updated weights for policy 1, policy_version 84540 (0.0009) [2023-10-12 23:34:59,338][44958] Updated weights for policy 0, policy_version 84130 (0.0008) [2023-10-12 23:34:59,714][44958] Updated weights for policy 0, policy_version 84140 (0.0008) [2023-10-12 23:35:00,082][44958] Updated weights for policy 0, policy_version 84150 (0.0007) [2023-10-12 23:35:00,446][44958] Updated weights for policy 0, policy_version 84160 (0.0007) [2023-10-12 23:35:00,682][44959] Updated weights for policy 1, policy_version 84550 (0.0007) [2023-10-12 23:35:01,048][44959] Updated weights for policy 1, policy_version 84560 (0.0009) [2023-10-12 23:35:01,416][44959] Updated weights for policy 1, policy_version 84570 (0.0008) [2023-10-12 23:35:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172752896. Throughput: 0: 1641.0, 1: 1642.8. Samples: 43201224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:35:01,443][43579] Avg episode reward: [(0, '283.060'), (1, '285.200')] [2023-10-12 23:35:04,475][44958] Updated weights for policy 0, policy_version 84170 (0.0008) [2023-10-12 23:35:04,842][44958] Updated weights for policy 0, policy_version 84180 (0.0009) [2023-10-12 23:35:05,224][44958] Updated weights for policy 0, policy_version 84190 (0.0008) [2023-10-12 23:35:05,716][44959] Updated weights for policy 1, policy_version 84580 (0.0007) [2023-10-12 23:35:06,086][44959] Updated weights for policy 1, policy_version 84590 (0.0008) [2023-10-12 23:35:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 172818432. Throughput: 0: 1641.2, 1: 1649.2. Samples: 43211808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:35:06,444][43579] Avg episode reward: [(0, '284.720'), (1, '277.190')] [2023-10-12 23:35:06,459][44959] Updated weights for policy 1, policy_version 84600 (0.0009) [2023-10-12 23:35:09,452][44958] Updated weights for policy 0, policy_version 84200 (0.0008) [2023-10-12 23:35:09,822][44958] Updated weights for policy 0, policy_version 84210 (0.0009) [2023-10-12 23:35:10,204][44958] Updated weights for policy 0, policy_version 84220 (0.0011) [2023-10-12 23:35:10,517][44959] Updated weights for policy 1, policy_version 84610 (0.0009) [2023-10-12 23:35:10,889][44959] Updated weights for policy 1, policy_version 84620 (0.0009) [2023-10-12 23:35:11,253][44959] Updated weights for policy 1, policy_version 84630 (0.0008) [2023-10-12 23:35:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172883968. Throughput: 0: 1636.4, 1: 1647.1. Samples: 43231012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:35:11,443][43579] Avg episode reward: [(0, '282.560'), (1, '276.200')] [2023-10-12 23:35:11,619][44959] Updated weights for policy 1, policy_version 84640 (0.0008) [2023-10-12 23:35:14,307][44958] Updated weights for policy 0, policy_version 84230 (0.0008) [2023-10-12 23:35:14,685][44958] Updated weights for policy 0, policy_version 84240 (0.0008) [2023-10-12 23:35:15,063][44958] Updated weights for policy 0, policy_version 84250 (0.0008) [2023-10-12 23:35:15,905][44959] Updated weights for policy 1, policy_version 84650 (0.0007) [2023-10-12 23:35:16,277][44959] Updated weights for policy 1, policy_version 84660 (0.0008) [2023-10-12 23:35:16,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 172949504. Throughput: 0: 1642.6, 1: 1641.8. Samples: 43250532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:35:16,443][43579] Avg episode reward: [(0, '275.100'), (1, '277.380')] [2023-10-12 23:35:16,638][44959] Updated weights for policy 1, policy_version 84670 (0.0009) [2023-10-12 23:35:19,051][44958] Updated weights for policy 0, policy_version 84260 (0.0007) [2023-10-12 23:35:19,419][44958] Updated weights for policy 0, policy_version 84270 (0.0007) [2023-10-12 23:35:19,789][44958] Updated weights for policy 0, policy_version 84280 (0.0008) [2023-10-12 23:35:20,978][44959] Updated weights for policy 1, policy_version 84680 (0.0008) [2023-10-12 23:35:21,344][44959] Updated weights for policy 1, policy_version 84690 (0.0008) [2023-10-12 23:35:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 173015040. Throughput: 0: 1641.9, 1: 1636.3. Samples: 43260960. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:35:21,443][43579] Avg episode reward: [(0, '270.240'), (1, '276.000')] [2023-10-12 23:35:21,713][44959] Updated weights for policy 1, policy_version 84700 (0.0009) [2023-10-12 23:35:24,261][44958] Updated weights for policy 0, policy_version 84290 (0.0008) [2023-10-12 23:35:24,635][44958] Updated weights for policy 0, policy_version 84300 (0.0009) [2023-10-12 23:35:25,007][44958] Updated weights for policy 0, policy_version 84310 (0.0010) [2023-10-12 23:35:25,377][44958] Updated weights for policy 0, policy_version 84320 (0.0008) [2023-10-12 23:35:25,664][44959] Updated weights for policy 1, policy_version 84710 (0.0010) [2023-10-12 23:35:26,029][44959] Updated weights for policy 1, policy_version 84720 (0.0009) [2023-10-12 23:35:26,408][44959] Updated weights for policy 1, policy_version 84730 (0.0009) [2023-10-12 23:35:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 173080576. Throughput: 0: 1636.2, 1: 1643.8. Samples: 43280532. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:35:26,443][43579] Avg episode reward: [(0, '268.520'), (1, '275.550')] [2023-10-12 23:35:29,755][44958] Updated weights for policy 0, policy_version 84330 (0.0010) [2023-10-12 23:35:30,129][44958] Updated weights for policy 0, policy_version 84340 (0.0007) [2023-10-12 23:35:30,500][44958] Updated weights for policy 0, policy_version 84350 (0.0007) [2023-10-12 23:35:30,833][44959] Updated weights for policy 1, policy_version 84740 (0.0008) [2023-10-12 23:35:31,221][44959] Updated weights for policy 1, policy_version 84750 (0.0007) [2023-10-12 23:35:31,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 173146112. Throughput: 0: 1642.9, 1: 1644.4. Samples: 43299840. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:35:31,444][43579] Avg episode reward: [(0, '273.930'), (1, '278.470')] [2023-10-12 23:35:31,592][44959] Updated weights for policy 1, policy_version 84760 (0.0008) [2023-10-12 23:35:34,458][44958] Updated weights for policy 0, policy_version 84360 (0.0007) [2023-10-12 23:35:34,833][44958] Updated weights for policy 0, policy_version 84370 (0.0009) [2023-10-12 23:35:35,206][44958] Updated weights for policy 0, policy_version 84380 (0.0009) [2023-10-12 23:35:35,741][44959] Updated weights for policy 1, policy_version 84770 (0.0008) [2023-10-12 23:35:36,103][44959] Updated weights for policy 1, policy_version 84780 (0.0008) [2023-10-12 23:35:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 173211648. Throughput: 0: 1645.9, 1: 1645.1. Samples: 43310320. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:35:36,443][43579] Avg episode reward: [(0, '268.340'), (1, '282.400')] [2023-10-12 23:35:36,475][44959] Updated weights for policy 1, policy_version 84790 (0.0008) [2023-10-12 23:35:36,842][44959] Updated weights for policy 1, policy_version 84800 (0.0008) [2023-10-12 23:35:39,681][44958] Updated weights for policy 0, policy_version 84390 (0.0009) [2023-10-12 23:35:40,058][44958] Updated weights for policy 0, policy_version 84400 (0.0011) [2023-10-12 23:35:40,437][44958] Updated weights for policy 0, policy_version 84410 (0.0009) [2023-10-12 23:35:40,747][44959] Updated weights for policy 1, policy_version 84810 (0.0007) [2023-10-12 23:35:41,120][44959] Updated weights for policy 1, policy_version 84820 (0.0008) [2023-10-12 23:35:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 173277184. Throughput: 0: 1638.7, 1: 1652.0. Samples: 43329866. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:35:41,444][43579] Avg episode reward: [(0, '276.020'), (1, '280.370')] [2023-10-12 23:35:41,486][44959] Updated weights for policy 1, policy_version 84830 (0.0010) [2023-10-12 23:35:44,476][44958] Updated weights for policy 0, policy_version 84420 (0.0008) [2023-10-12 23:35:44,856][44958] Updated weights for policy 0, policy_version 84430 (0.0009) [2023-10-12 23:35:45,226][44958] Updated weights for policy 0, policy_version 84440 (0.0010) [2023-10-12 23:35:45,744][44959] Updated weights for policy 1, policy_version 84840 (0.0009) [2023-10-12 23:35:46,126][44959] Updated weights for policy 1, policy_version 84850 (0.0007) [2023-10-12 23:35:46,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 173342720. Throughput: 0: 1639.3, 1: 1645.4. Samples: 43349034. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:35:46,444][43579] Avg episode reward: [(0, '276.080'), (1, '281.330')] [2023-10-12 23:35:46,454][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000084448_86474752.pth... [2023-10-12 23:35:46,490][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000082912_84901888.pth [2023-10-12 23:35:46,494][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000084448_86474752.pth [2023-10-12 23:35:46,494][44959] Updated weights for policy 1, policy_version 84860 (0.0008) [2023-10-12 23:35:46,632][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000084864_86900736.pth... [2023-10-12 23:35:46,671][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000083328_85327872.pth [2023-10-12 23:35:46,676][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000084864_86900736.pth [2023-10-12 23:35:49,266][44958] Updated weights for policy 0, policy_version 84450 (0.0008) [2023-10-12 23:35:49,631][44958] Updated weights for policy 0, policy_version 84460 (0.0009) [2023-10-12 23:35:50,002][44958] Updated weights for policy 0, policy_version 84470 (0.0009) [2023-10-12 23:35:50,380][44958] Updated weights for policy 0, policy_version 84480 (0.0009) [2023-10-12 23:35:50,603][44959] Updated weights for policy 1, policy_version 84870 (0.0009) [2023-10-12 23:35:50,971][44959] Updated weights for policy 1, policy_version 84880 (0.0009) [2023-10-12 23:35:51,341][44959] Updated weights for policy 1, policy_version 84890 (0.0008) [2023-10-12 23:35:51,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 173408256. Throughput: 0: 1637.2, 1: 1647.4. Samples: 43359612. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:35:51,444][43579] Avg episode reward: [(0, '280.340'), (1, '280.260')] [2023-10-12 23:35:54,489][44958] Updated weights for policy 0, policy_version 84490 (0.0011) [2023-10-12 23:35:54,858][44958] Updated weights for policy 0, policy_version 84500 (0.0009) [2023-10-12 23:35:55,228][44958] Updated weights for policy 0, policy_version 84510 (0.0009) [2023-10-12 23:35:55,373][44959] Updated weights for policy 1, policy_version 84900 (0.0007) [2023-10-12 23:35:55,741][44959] Updated weights for policy 1, policy_version 84910 (0.0007) [2023-10-12 23:35:56,103][44959] Updated weights for policy 1, policy_version 84920 (0.0009) [2023-10-12 23:35:56,443][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 173506560. Throughput: 0: 1635.5, 1: 1651.4. Samples: 43378926. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:35:56,444][43579] Avg episode reward: [(0, '278.880'), (1, '278.120')] [2023-10-12 23:35:59,418][44958] Updated weights for policy 0, policy_version 84520 (0.0010) [2023-10-12 23:35:59,803][44958] Updated weights for policy 0, policy_version 84530 (0.0008) [2023-10-12 23:36:00,163][44958] Updated weights for policy 0, policy_version 84540 (0.0009) [2023-10-12 23:36:00,202][44959] Updated weights for policy 1, policy_version 84930 (0.0008) [2023-10-12 23:36:00,570][44959] Updated weights for policy 1, policy_version 84940 (0.0010) [2023-10-12 23:36:00,936][44959] Updated weights for policy 1, policy_version 84950 (0.0011) [2023-10-12 23:36:01,301][44959] Updated weights for policy 1, policy_version 84960 (0.0010) [2023-10-12 23:36:01,443][43579] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 173572096. Throughput: 0: 1638.0, 1: 1642.4. Samples: 43398154. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:36:01,443][43579] Avg episode reward: [(0, '275.530'), (1, '277.870')] [2023-10-12 23:36:04,468][44958] Updated weights for policy 0, policy_version 84550 (0.0008) [2023-10-12 23:36:04,837][44958] Updated weights for policy 0, policy_version 84560 (0.0009) [2023-10-12 23:36:05,214][44958] Updated weights for policy 0, policy_version 84570 (0.0010) [2023-10-12 23:36:05,397][44959] Updated weights for policy 1, policy_version 84970 (0.0008) [2023-10-12 23:36:05,771][44959] Updated weights for policy 1, policy_version 84980 (0.0009) [2023-10-12 23:36:06,134][44959] Updated weights for policy 1, policy_version 84990 (0.0008) [2023-10-12 23:36:06,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 173637632. Throughput: 0: 1636.4, 1: 1653.6. Samples: 43409006. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:36:06,443][43579] Avg episode reward: [(0, '276.890'), (1, '276.770')] [2023-10-12 23:36:09,620][44958] Updated weights for policy 0, policy_version 84580 (0.0009) [2023-10-12 23:36:09,990][44958] Updated weights for policy 0, policy_version 84590 (0.0008) [2023-10-12 23:36:10,350][44958] Updated weights for policy 0, policy_version 84600 (0.0008) [2023-10-12 23:36:10,446][44959] Updated weights for policy 1, policy_version 85000 (0.0008) [2023-10-12 23:36:10,803][44959] Updated weights for policy 1, policy_version 85010 (0.0009) [2023-10-12 23:36:11,173][44959] Updated weights for policy 1, policy_version 85020 (0.0008) [2023-10-12 23:36:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 173703168. Throughput: 0: 1636.1, 1: 1648.9. Samples: 43428356. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-12 23:36:11,444][43579] Avg episode reward: [(0, '278.670'), (1, '279.710')] [2023-10-12 23:36:14,605][44958] Updated weights for policy 0, policy_version 84610 (0.0008) [2023-10-12 23:36:15,014][44958] Updated weights for policy 0, policy_version 84620 (0.0010) [2023-10-12 23:36:15,345][44959] Updated weights for policy 1, policy_version 85030 (0.0009) [2023-10-12 23:36:15,395][44958] Updated weights for policy 0, policy_version 84630 (0.0007) [2023-10-12 23:36:15,733][44959] Updated weights for policy 1, policy_version 85040 (0.0008) [2023-10-12 23:36:15,764][44958] Updated weights for policy 0, policy_version 84640 (0.0008) [2023-10-12 23:36:16,102][44959] Updated weights for policy 1, policy_version 85050 (0.0007) [2023-10-12 23:36:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 173768704. Throughput: 0: 1634.4, 1: 1640.9. Samples: 43447228. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:36:16,444][43579] Avg episode reward: [(0, '278.390'), (1, '279.300')] [2023-10-12 23:36:19,760][44958] Updated weights for policy 0, policy_version 84650 (0.0007) [2023-10-12 23:36:20,139][44958] Updated weights for policy 0, policy_version 84660 (0.0007) [2023-10-12 23:36:20,142][44959] Updated weights for policy 1, policy_version 85060 (0.0009) [2023-10-12 23:36:20,509][44959] Updated weights for policy 1, policy_version 85070 (0.0010) [2023-10-12 23:36:20,509][44958] Updated weights for policy 0, policy_version 84670 (0.0010) [2023-10-12 23:36:20,880][44959] Updated weights for policy 1, policy_version 85080 (0.0010) [2023-10-12 23:36:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 173834240. Throughput: 0: 1630.5, 1: 1653.8. Samples: 43458114. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:36:21,443][43579] Avg episode reward: [(0, '280.290'), (1, '282.220')] [2023-10-12 23:36:24,493][44958] Updated weights for policy 0, policy_version 84680 (0.0007) [2023-10-12 23:36:24,868][44958] Updated weights for policy 0, policy_version 84690 (0.0008) [2023-10-12 23:36:24,998][44959] Updated weights for policy 1, policy_version 85090 (0.0010) [2023-10-12 23:36:25,244][44958] Updated weights for policy 0, policy_version 84700 (0.0007) [2023-10-12 23:36:25,371][44959] Updated weights for policy 1, policy_version 85100 (0.0008) [2023-10-12 23:36:25,747][44959] Updated weights for policy 1, policy_version 85110 (0.0009) [2023-10-12 23:36:26,116][44959] Updated weights for policy 1, policy_version 85120 (0.0010) [2023-10-12 23:36:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 173899776. Throughput: 0: 1631.7, 1: 1645.4. Samples: 43477336. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:36:26,443][43579] Avg episode reward: [(0, '283.530'), (1, '281.870')] [2023-10-12 23:36:29,480][44958] Updated weights for policy 0, policy_version 84710 (0.0008) [2023-10-12 23:36:29,853][44958] Updated weights for policy 0, policy_version 84720 (0.0007) [2023-10-12 23:36:30,221][44958] Updated weights for policy 0, policy_version 84730 (0.0008) [2023-10-12 23:36:30,308][44959] Updated weights for policy 1, policy_version 85130 (0.0008) [2023-10-12 23:36:30,681][44959] Updated weights for policy 1, policy_version 85140 (0.0007) [2023-10-12 23:36:31,053][44959] Updated weights for policy 1, policy_version 85150 (0.0008) [2023-10-12 23:36:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 173965312. Throughput: 0: 1633.1, 1: 1642.4. Samples: 43496430. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:36:31,443][43579] Avg episode reward: [(0, '282.460'), (1, '281.900')] [2023-10-12 23:36:34,454][44958] Updated weights for policy 0, policy_version 84740 (0.0007) [2023-10-12 23:36:34,826][44958] Updated weights for policy 0, policy_version 84750 (0.0008) [2023-10-12 23:36:35,080][44959] Updated weights for policy 1, policy_version 85160 (0.0008) [2023-10-12 23:36:35,196][44958] Updated weights for policy 0, policy_version 84760 (0.0009) [2023-10-12 23:36:35,443][44959] Updated weights for policy 1, policy_version 85170 (0.0007) [2023-10-12 23:36:35,819][44959] Updated weights for policy 1, policy_version 85180 (0.0008) [2023-10-12 23:36:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 174030848. Throughput: 0: 1636.1, 1: 1656.3. Samples: 43507768. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:36:36,443][43579] Avg episode reward: [(0, '283.250'), (1, '284.730')] [2023-10-12 23:36:39,217][44958] Updated weights for policy 0, policy_version 84770 (0.0009) [2023-10-12 23:36:39,592][44958] Updated weights for policy 0, policy_version 84780 (0.0009) [2023-10-12 23:36:39,881][44959] Updated weights for policy 1, policy_version 85190 (0.0007) [2023-10-12 23:36:39,970][44958] Updated weights for policy 0, policy_version 84790 (0.0008) [2023-10-12 23:36:40,251][44959] Updated weights for policy 1, policy_version 85200 (0.0007) [2023-10-12 23:36:40,337][44958] Updated weights for policy 0, policy_version 84800 (0.0008) [2023-10-12 23:36:40,624][44959] Updated weights for policy 1, policy_version 85210 (0.0007) [2023-10-12 23:36:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 174096384. Throughput: 0: 1639.4, 1: 1642.2. Samples: 43526600. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:36:41,443][43579] Avg episode reward: [(0, '281.400'), (1, '281.300')] [2023-10-12 23:36:44,447][44958] Updated weights for policy 0, policy_version 84810 (0.0009) [2023-10-12 23:36:44,825][44958] Updated weights for policy 0, policy_version 84820 (0.0007) [2023-10-12 23:36:44,949][44959] Updated weights for policy 1, policy_version 85220 (0.0007) [2023-10-12 23:36:45,191][44958] Updated weights for policy 0, policy_version 84830 (0.0007) [2023-10-12 23:36:45,314][44959] Updated weights for policy 1, policy_version 85230 (0.0008) [2023-10-12 23:36:45,677][44959] Updated weights for policy 1, policy_version 85240 (0.0009) [2023-10-12 23:36:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 174161920. Throughput: 0: 1638.0, 1: 1642.9. Samples: 43545798. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:36:46,443][43579] Avg episode reward: [(0, '280.170'), (1, '280.890')] [2023-10-12 23:36:49,373][44958] Updated weights for policy 0, policy_version 84840 (0.0008) [2023-10-12 23:36:49,747][44958] Updated weights for policy 0, policy_version 84850 (0.0009) [2023-10-12 23:36:49,883][44959] Updated weights for policy 1, policy_version 85250 (0.0010) [2023-10-12 23:36:50,121][44958] Updated weights for policy 0, policy_version 84860 (0.0009) [2023-10-12 23:36:50,254][44959] Updated weights for policy 1, policy_version 85260 (0.0009) [2023-10-12 23:36:50,612][44959] Updated weights for policy 1, policy_version 85270 (0.0010) [2023-10-12 23:36:50,979][44959] Updated weights for policy 1, policy_version 85280 (0.0011) [2023-10-12 23:36:51,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 174227456. Throughput: 0: 1636.4, 1: 1648.3. Samples: 43556816. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:36:51,443][43579] Avg episode reward: [(0, '275.390'), (1, '282.140')] [2023-10-12 23:36:54,195][44958] Updated weights for policy 0, policy_version 84870 (0.0008) [2023-10-12 23:36:54,579][44958] Updated weights for policy 0, policy_version 84880 (0.0011) [2023-10-12 23:36:54,947][44958] Updated weights for policy 0, policy_version 84890 (0.0011) [2023-10-12 23:36:55,250][44959] Updated weights for policy 1, policy_version 85290 (0.0007) [2023-10-12 23:36:55,617][44959] Updated weights for policy 1, policy_version 85300 (0.0010) [2023-10-12 23:36:55,992][44959] Updated weights for policy 1, policy_version 85310 (0.0009) [2023-10-12 23:36:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 174292992. Throughput: 0: 1638.7, 1: 1642.9. Samples: 43576026. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:36:56,443][43579] Avg episode reward: [(0, '271.810'), (1, '281.530')] [2023-10-12 23:36:59,549][44958] Updated weights for policy 0, policy_version 84900 (0.0008) [2023-10-12 23:36:59,945][44958] Updated weights for policy 0, policy_version 84910 (0.0008) [2023-10-12 23:37:00,119][44959] Updated weights for policy 1, policy_version 85320 (0.0009) [2023-10-12 23:37:00,312][44958] Updated weights for policy 0, policy_version 84920 (0.0008) [2023-10-12 23:37:00,490][44959] Updated weights for policy 1, policy_version 85330 (0.0009) [2023-10-12 23:37:00,861][44959] Updated weights for policy 1, policy_version 85340 (0.0010) [2023-10-12 23:37:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 174358528. Throughput: 0: 1643.6, 1: 1636.6. Samples: 43594840. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:37:01,443][43579] Avg episode reward: [(0, '272.560'), (1, '284.810')] [2023-10-12 23:37:04,478][44958] Updated weights for policy 0, policy_version 84930 (0.0011) [2023-10-12 23:37:04,863][44958] Updated weights for policy 0, policy_version 84940 (0.0008) [2023-10-12 23:37:05,196][44959] Updated weights for policy 1, policy_version 85350 (0.0008) [2023-10-12 23:37:05,230][44958] Updated weights for policy 0, policy_version 84950 (0.0007) [2023-10-12 23:37:05,565][44959] Updated weights for policy 1, policy_version 85360 (0.0009) [2023-10-12 23:37:05,593][44958] Updated weights for policy 0, policy_version 84960 (0.0008) [2023-10-12 23:37:05,922][44959] Updated weights for policy 1, policy_version 85370 (0.0009) [2023-10-12 23:37:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 174424064. Throughput: 0: 1641.4, 1: 1646.5. Samples: 43606070. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) [2023-10-12 23:37:06,443][43579] Avg episode reward: [(0, '270.570'), (1, '287.490')] [2023-10-12 23:37:09,689][44958] Updated weights for policy 0, policy_version 84970 (0.0009) [2023-10-12 23:37:10,062][44958] Updated weights for policy 0, policy_version 84980 (0.0009) [2023-10-12 23:37:10,085][44959] Updated weights for policy 1, policy_version 85380 (0.0009) [2023-10-12 23:37:10,432][44958] Updated weights for policy 0, policy_version 84990 (0.0009) [2023-10-12 23:37:10,457][44959] Updated weights for policy 1, policy_version 85390 (0.0007) [2023-10-12 23:37:10,817][44959] Updated weights for policy 1, policy_version 85400 (0.0007) [2023-10-12 23:37:11,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 174489600. Throughput: 0: 1647.2, 1: 1647.2. Samples: 43625588. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:11,443][43579] Avg episode reward: [(0, '271.990'), (1, '284.810')] [2023-10-12 23:37:14,748][44958] Updated weights for policy 0, policy_version 85000 (0.0008) [2023-10-12 23:37:14,758][44959] Updated weights for policy 1, policy_version 85410 (0.0008) [2023-10-12 23:37:15,122][44958] Updated weights for policy 0, policy_version 85010 (0.0009) [2023-10-12 23:37:15,123][44959] Updated weights for policy 1, policy_version 85420 (0.0008) [2023-10-12 23:37:15,489][44958] Updated weights for policy 0, policy_version 85020 (0.0010) [2023-10-12 23:37:15,496][44959] Updated weights for policy 1, policy_version 85430 (0.0009) [2023-10-12 23:37:15,860][44959] Updated weights for policy 1, policy_version 85440 (0.0008) [2023-10-12 23:37:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 174555136. Throughput: 0: 1639.0, 1: 1647.4. Samples: 43644316. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:16,444][43579] Avg episode reward: [(0, '271.830'), (1, '281.840')] [2023-10-12 23:37:19,496][44958] Updated weights for policy 0, policy_version 85030 (0.0008) [2023-10-12 23:37:19,864][44958] Updated weights for policy 0, policy_version 85040 (0.0007) [2023-10-12 23:37:20,093][44959] Updated weights for policy 1, policy_version 85450 (0.0008) [2023-10-12 23:37:20,248][44958] Updated weights for policy 0, policy_version 85050 (0.0010) [2023-10-12 23:37:20,461][44959] Updated weights for policy 1, policy_version 85460 (0.0008) [2023-10-12 23:37:20,831][44959] Updated weights for policy 1, policy_version 85470 (0.0010) [2023-10-12 23:37:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 174620672. Throughput: 0: 1641.3, 1: 1645.7. Samples: 43655684. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:21,443][43579] Avg episode reward: [(0, '271.700'), (1, '280.700')] [2023-10-12 23:37:24,389][44958] Updated weights for policy 0, policy_version 85060 (0.0008) [2023-10-12 23:37:24,759][44958] Updated weights for policy 0, policy_version 85070 (0.0010) [2023-10-12 23:37:24,824][44959] Updated weights for policy 1, policy_version 85480 (0.0008) [2023-10-12 23:37:25,127][44958] Updated weights for policy 0, policy_version 85080 (0.0009) [2023-10-12 23:37:25,193][44959] Updated weights for policy 1, policy_version 85490 (0.0009) [2023-10-12 23:37:25,556][44959] Updated weights for policy 1, policy_version 85500 (0.0008) [2023-10-12 23:37:26,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 174686208. Throughput: 0: 1640.2, 1: 1642.9. Samples: 43674342. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:26,443][43579] Avg episode reward: [(0, '277.770'), (1, '278.390')] [2023-10-12 23:37:29,715][44958] Updated weights for policy 0, policy_version 85090 (0.0007) [2023-10-12 23:37:30,076][44958] Updated weights for policy 0, policy_version 85100 (0.0010) [2023-10-12 23:37:30,105][44959] Updated weights for policy 1, policy_version 85510 (0.0009) [2023-10-12 23:37:30,443][44958] Updated weights for policy 0, policy_version 85110 (0.0010) [2023-10-12 23:37:30,472][44959] Updated weights for policy 1, policy_version 85520 (0.0011) [2023-10-12 23:37:30,818][44958] Updated weights for policy 0, policy_version 85120 (0.0008) [2023-10-12 23:37:30,839][44959] Updated weights for policy 1, policy_version 85530 (0.0009) [2023-10-12 23:37:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 174751744. Throughput: 0: 1619.6, 1: 1633.9. Samples: 43692206. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:31,443][43579] Avg episode reward: [(0, '278.580'), (1, '278.950')] [2023-10-12 23:37:35,039][44958] Updated weights for policy 0, policy_version 85130 (0.0008) [2023-10-12 23:37:35,257][44959] Updated weights for policy 1, policy_version 85540 (0.0008) [2023-10-12 23:37:35,406][44958] Updated weights for policy 0, policy_version 85140 (0.0010) [2023-10-12 23:37:35,615][44959] Updated weights for policy 1, policy_version 85550 (0.0008) [2023-10-12 23:37:35,773][44958] Updated weights for policy 0, policy_version 85150 (0.0008) [2023-10-12 23:37:35,988][44959] Updated weights for policy 1, policy_version 85560 (0.0009) [2023-10-12 23:37:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 174817280. Throughput: 0: 1626.0, 1: 1628.8. Samples: 43703282. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:36,443][43579] Avg episode reward: [(0, '279.210'), (1, '278.050')] [2023-10-12 23:37:40,297][44959] Updated weights for policy 1, policy_version 85570 (0.0009) [2023-10-12 23:37:40,460][44958] Updated weights for policy 0, policy_version 85160 (0.0009) [2023-10-12 23:37:40,662][44959] Updated weights for policy 1, policy_version 85580 (0.0010) [2023-10-12 23:37:40,827][44958] Updated weights for policy 0, policy_version 85170 (0.0009) [2023-10-12 23:37:41,034][44959] Updated weights for policy 1, policy_version 85590 (0.0008) [2023-10-12 23:37:41,205][44958] Updated weights for policy 0, policy_version 85180 (0.0010) [2023-10-12 23:37:41,402][44959] Updated weights for policy 1, policy_version 85600 (0.0008) [2023-10-12 23:37:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 174882816. Throughput: 0: 1623.2, 1: 1623.0. Samples: 43722106. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:41,443][43579] Avg episode reward: [(0, '286.140'), (1, '282.990')] [2023-10-12 23:37:45,913][44958] Updated weights for policy 0, policy_version 85190 (0.0010) [2023-10-12 23:37:46,003][44959] Updated weights for policy 1, policy_version 85610 (0.0009) [2023-10-12 23:37:46,293][44958] Updated weights for policy 0, policy_version 85200 (0.0009) [2023-10-12 23:37:46,374][44959] Updated weights for policy 1, policy_version 85620 (0.0009) [2023-10-12 23:37:46,442][43579] Fps is (10 sec: 6553.6, 60 sec: 12015.0, 300 sec: 12996.1). Total num frames: 174882816. Throughput: 0: 1597.6, 1: 1618.0. Samples: 43739544. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:46,443][43579] Avg episode reward: [(0, '286.300'), (1, '285.480')] [2023-10-12 23:37:46,654][44958] Updated weights for policy 0, policy_version 85210 (0.0010) [2023-10-12 23:37:46,734][44959] Updated weights for policy 1, policy_version 85630 (0.0009) [2023-10-12 23:37:46,805][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000085632_87687168.pth... [2023-10-12 23:37:46,835][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000084096_86114304.pth [2023-10-12 23:37:46,874][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000085216_87261184.pth... [2023-10-12 23:37:46,912][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000083680_85688320.pth [2023-10-12 23:37:51,124][44958] Updated weights for policy 0, policy_version 85220 (0.0009) [2023-10-12 23:37:51,443][43579] Fps is (10 sec: 6553.5, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 174948352. Throughput: 0: 1573.8, 1: 1586.7. Samples: 43748292. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:51,444][43579] Avg episode reward: [(0, '287.580'), (1, '287.900')] [2023-10-12 23:37:51,488][44958] Updated weights for policy 0, policy_version 85230 (0.0009) [2023-10-12 23:37:51,493][44959] Updated weights for policy 1, policy_version 85640 (0.0008) [2023-10-12 23:37:51,860][44958] Updated weights for policy 0, policy_version 85240 (0.0009) [2023-10-12 23:37:51,864][44959] Updated weights for policy 1, policy_version 85650 (0.0009) [2023-10-12 23:37:52,231][44959] Updated weights for policy 1, policy_version 85660 (0.0009) [2023-10-12 23:37:56,263][44958] Updated weights for policy 0, policy_version 85250 (0.0009) [2023-10-12 23:37:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 175013888. Throughput: 0: 1575.5, 1: 1568.5. Samples: 43767068. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:37:56,444][43579] Avg episode reward: [(0, '290.860'), (1, '282.430')] [2023-10-12 23:37:56,622][44958] Updated weights for policy 0, policy_version 85260 (0.0010) [2023-10-12 23:37:56,712][44959] Updated weights for policy 1, policy_version 85670 (0.0009) [2023-10-12 23:37:56,994][44958] Updated weights for policy 0, policy_version 85270 (0.0008) [2023-10-12 23:37:57,073][44959] Updated weights for policy 1, policy_version 85680 (0.0009) [2023-10-12 23:37:57,359][44518] Saving new best policy, reward=290.860! [2023-10-12 23:37:57,365][44958] Updated weights for policy 0, policy_version 85280 (0.0009) [2023-10-12 23:37:57,436][44959] Updated weights for policy 1, policy_version 85690 (0.0009) [2023-10-12 23:38:01,442][43579] Fps is (10 sec: 13107.4, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 175079424. Throughput: 0: 1572.1, 1: 1572.2. Samples: 43785808. Policy #0 lag: (min: 11.0, avg: 26.9, max: 43.0) [2023-10-12 23:38:01,443][43579] Avg episode reward: [(0, '289.030'), (1, '283.260')] [2023-10-12 23:38:01,854][44959] Updated weights for policy 1, policy_version 85700 (0.0008) [2023-10-12 23:38:02,067][44958] Updated weights for policy 0, policy_version 85290 (0.0009) [2023-10-12 23:38:02,226][44959] Updated weights for policy 1, policy_version 85710 (0.0009) [2023-10-12 23:38:02,438][44958] Updated weights for policy 0, policy_version 85300 (0.0009) [2023-10-12 23:38:02,594][44959] Updated weights for policy 1, policy_version 85720 (0.0009) [2023-10-12 23:38:02,813][44958] Updated weights for policy 0, policy_version 85310 (0.0010) [2023-10-12 23:38:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 175144960. Throughput: 0: 1535.9, 1: 1540.1. Samples: 43794104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:06,444][43579] Avg episode reward: [(0, '285.530'), (1, '283.410')] [2023-10-12 23:38:06,953][44959] Updated weights for policy 1, policy_version 85730 (0.0009) [2023-10-12 23:38:07,042][44958] Updated weights for policy 0, policy_version 85320 (0.0009) [2023-10-12 23:38:07,325][44959] Updated weights for policy 1, policy_version 85740 (0.0008) [2023-10-12 23:38:07,412][44958] Updated weights for policy 0, policy_version 85330 (0.0007) [2023-10-12 23:38:07,692][44959] Updated weights for policy 1, policy_version 85750 (0.0008) [2023-10-12 23:38:07,788][44958] Updated weights for policy 0, policy_version 85340 (0.0007) [2023-10-12 23:38:08,056][44959] Updated weights for policy 1, policy_version 85760 (0.0009) [2023-10-12 23:38:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 175210496. Throughput: 0: 1553.1, 1: 1552.5. Samples: 43814096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:11,443][43579] Avg episode reward: [(0, '283.850'), (1, '281.260')] [2023-10-12 23:38:12,150][44958] Updated weights for policy 0, policy_version 85350 (0.0007) [2023-10-12 23:38:12,370][44959] Updated weights for policy 1, policy_version 85770 (0.0007) [2023-10-12 23:38:12,527][44958] Updated weights for policy 0, policy_version 85360 (0.0008) [2023-10-12 23:38:12,744][44959] Updated weights for policy 1, policy_version 85780 (0.0008) [2023-10-12 23:38:12,902][44958] Updated weights for policy 0, policy_version 85370 (0.0009) [2023-10-12 23:38:13,106][44959] Updated weights for policy 1, policy_version 85790 (0.0009) [2023-10-12 23:38:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 12015.0, 300 sec: 12996.1). Total num frames: 175276032. Throughput: 0: 1578.2, 1: 1577.4. Samples: 43834208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:16,444][43579] Avg episode reward: [(0, '284.030'), (1, '278.420')] [2023-10-12 23:38:16,955][44958] Updated weights for policy 0, policy_version 85380 (0.0009) [2023-10-12 23:38:17,325][44958] Updated weights for policy 0, policy_version 85390 (0.0007) [2023-10-12 23:38:17,371][44959] Updated weights for policy 1, policy_version 85800 (0.0008) [2023-10-12 23:38:17,691][44958] Updated weights for policy 0, policy_version 85400 (0.0008) [2023-10-12 23:38:17,749][44959] Updated weights for policy 1, policy_version 85810 (0.0008) [2023-10-12 23:38:18,120][44959] Updated weights for policy 1, policy_version 85820 (0.0009) [2023-10-12 23:38:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 175341568. Throughput: 0: 1554.1, 1: 1552.5. Samples: 43843082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:21,444][43579] Avg episode reward: [(0, '277.990'), (1, '276.670')] [2023-10-12 23:38:21,850][44958] Updated weights for policy 0, policy_version 85410 (0.0007) [2023-10-12 23:38:22,215][44959] Updated weights for policy 1, policy_version 85830 (0.0009) [2023-10-12 23:38:22,224][44958] Updated weights for policy 0, policy_version 85420 (0.0008) [2023-10-12 23:38:22,586][44959] Updated weights for policy 1, policy_version 85840 (0.0009) [2023-10-12 23:38:22,590][44958] Updated weights for policy 0, policy_version 85430 (0.0007) [2023-10-12 23:38:22,959][44959] Updated weights for policy 1, policy_version 85850 (0.0008) [2023-10-12 23:38:22,966][44958] Updated weights for policy 0, policy_version 85440 (0.0008) [2023-10-12 23:38:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 175407104. Throughput: 0: 1567.7, 1: 1572.4. Samples: 43863412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:26,444][43579] Avg episode reward: [(0, '280.030'), (1, '282.990')] [2023-10-12 23:38:26,968][44959] Updated weights for policy 1, policy_version 85860 (0.0009) [2023-10-12 23:38:27,298][44958] Updated weights for policy 0, policy_version 85450 (0.0007) [2023-10-12 23:38:27,331][44959] Updated weights for policy 1, policy_version 85870 (0.0007) [2023-10-12 23:38:27,670][44958] Updated weights for policy 0, policy_version 85460 (0.0009) [2023-10-12 23:38:27,698][44959] Updated weights for policy 1, policy_version 85880 (0.0007) [2023-10-12 23:38:28,030][44958] Updated weights for policy 0, policy_version 85470 (0.0010) [2023-10-12 23:38:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 175472640. Throughput: 0: 1601.1, 1: 1607.2. Samples: 43883918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:31,443][43579] Avg episode reward: [(0, '272.340'), (1, '287.310')] [2023-10-12 23:38:31,907][44959] Updated weights for policy 1, policy_version 85890 (0.0010) [2023-10-12 23:38:32,214][44958] Updated weights for policy 0, policy_version 85480 (0.0007) [2023-10-12 23:38:32,274][44959] Updated weights for policy 1, policy_version 85900 (0.0007) [2023-10-12 23:38:32,582][44958] Updated weights for policy 0, policy_version 85490 (0.0007) [2023-10-12 23:38:32,643][44959] Updated weights for policy 1, policy_version 85910 (0.0008) [2023-10-12 23:38:32,949][44958] Updated weights for policy 0, policy_version 85500 (0.0008) [2023-10-12 23:38:33,000][44959] Updated weights for policy 1, policy_version 85920 (0.0009) [2023-10-12 23:38:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 175538176. Throughput: 0: 1596.2, 1: 1609.8. Samples: 43892564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:36,444][43579] Avg episode reward: [(0, '274.660'), (1, '289.390')] [2023-10-12 23:38:37,122][44958] Updated weights for policy 0, policy_version 85510 (0.0009) [2023-10-12 23:38:37,246][44959] Updated weights for policy 1, policy_version 85930 (0.0007) [2023-10-12 23:38:37,498][44958] Updated weights for policy 0, policy_version 85520 (0.0008) [2023-10-12 23:38:37,608][44959] Updated weights for policy 1, policy_version 85940 (0.0007) [2023-10-12 23:38:37,879][44958] Updated weights for policy 0, policy_version 85530 (0.0008) [2023-10-12 23:38:37,964][44959] Updated weights for policy 1, policy_version 85950 (0.0009) [2023-10-12 23:38:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 175603712. Throughput: 0: 1604.0, 1: 1628.1. Samples: 43912512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:41,443][43579] Avg episode reward: [(0, '274.370'), (1, '292.010')] [2023-10-12 23:38:42,129][44958] Updated weights for policy 0, policy_version 85540 (0.0008) [2023-10-12 23:38:42,196][44959] Updated weights for policy 1, policy_version 85960 (0.0008) [2023-10-12 23:38:42,496][44958] Updated weights for policy 0, policy_version 85550 (0.0009) [2023-10-12 23:38:42,571][44959] Updated weights for policy 1, policy_version 85970 (0.0007) [2023-10-12 23:38:42,874][44958] Updated weights for policy 0, policy_version 85560 (0.0009) [2023-10-12 23:38:42,934][44959] Updated weights for policy 1, policy_version 85980 (0.0007) [2023-10-12 23:38:46,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 175669248. Throughput: 0: 1622.7, 1: 1641.2. Samples: 43932686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:46,444][43579] Avg episode reward: [(0, '278.280'), (1, '291.320')] [2023-10-12 23:38:47,061][44958] Updated weights for policy 0, policy_version 85570 (0.0009) [2023-10-12 23:38:47,099][44959] Updated weights for policy 1, policy_version 85990 (0.0009) [2023-10-12 23:38:47,426][44958] Updated weights for policy 0, policy_version 85580 (0.0008) [2023-10-12 23:38:47,464][44959] Updated weights for policy 1, policy_version 86000 (0.0008) [2023-10-12 23:38:47,800][44958] Updated weights for policy 0, policy_version 85590 (0.0007) [2023-10-12 23:38:47,832][44959] Updated weights for policy 1, policy_version 86010 (0.0009) [2023-10-12 23:38:48,167][44958] Updated weights for policy 0, policy_version 85600 (0.0010) [2023-10-12 23:38:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 175734784. Throughput: 0: 1627.4, 1: 1644.9. Samples: 43941356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:51,443][43579] Avg episode reward: [(0, '278.660'), (1, '294.040')] [2023-10-12 23:38:51,444][44583] Saving new best policy, reward=294.040! [2023-10-12 23:38:52,072][44959] Updated weights for policy 1, policy_version 86020 (0.0010) [2023-10-12 23:38:52,438][44959] Updated weights for policy 1, policy_version 86030 (0.0008) [2023-10-12 23:38:52,550][44958] Updated weights for policy 0, policy_version 85610 (0.0008) [2023-10-12 23:38:52,805][44959] Updated weights for policy 1, policy_version 86040 (0.0008) [2023-10-12 23:38:52,912][44958] Updated weights for policy 0, policy_version 85620 (0.0008) [2023-10-12 23:38:53,290][44958] Updated weights for policy 0, policy_version 85630 (0.0010) [2023-10-12 23:38:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 175800320. Throughput: 0: 1631.8, 1: 1649.7. Samples: 43961766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:38:56,443][43579] Avg episode reward: [(0, '270.050'), (1, '292.620')] [2023-10-12 23:38:56,839][44959] Updated weights for policy 1, policy_version 86050 (0.0007) [2023-10-12 23:38:57,211][44959] Updated weights for policy 1, policy_version 86060 (0.0007) [2023-10-12 23:38:57,367][44958] Updated weights for policy 0, policy_version 85640 (0.0009) [2023-10-12 23:38:57,571][44959] Updated weights for policy 1, policy_version 86070 (0.0009) [2023-10-12 23:38:57,743][44958] Updated weights for policy 0, policy_version 85650 (0.0008) [2023-10-12 23:38:57,937][44959] Updated weights for policy 1, policy_version 86080 (0.0009) [2023-10-12 23:38:58,117][44958] Updated weights for policy 0, policy_version 85660 (0.0009) [2023-10-12 23:39:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 175865856. Throughput: 0: 1631.3, 1: 1655.7. Samples: 43982126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:01,443][43579] Avg episode reward: [(0, '274.040'), (1, '290.750')] [2023-10-12 23:39:02,045][44959] Updated weights for policy 1, policy_version 86090 (0.0008) [2023-10-12 23:39:02,250][44958] Updated weights for policy 0, policy_version 85670 (0.0008) [2023-10-12 23:39:02,410][44959] Updated weights for policy 1, policy_version 86100 (0.0009) [2023-10-12 23:39:02,610][44958] Updated weights for policy 0, policy_version 85680 (0.0008) [2023-10-12 23:39:02,779][44959] Updated weights for policy 1, policy_version 86110 (0.0007) [2023-10-12 23:39:02,979][44958] Updated weights for policy 0, policy_version 85690 (0.0008) [2023-10-12 23:39:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 175931392. Throughput: 0: 1623.9, 1: 1663.2. Samples: 43990998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:06,444][43579] Avg episode reward: [(0, '273.510'), (1, '290.040')] [2023-10-12 23:39:06,818][44959] Updated weights for policy 1, policy_version 86120 (0.0008) [2023-10-12 23:39:07,184][44959] Updated weights for policy 1, policy_version 86130 (0.0008) [2023-10-12 23:39:07,273][44958] Updated weights for policy 0, policy_version 85700 (0.0009) [2023-10-12 23:39:07,560][44959] Updated weights for policy 1, policy_version 86140 (0.0007) [2023-10-12 23:39:07,639][44958] Updated weights for policy 0, policy_version 85710 (0.0009) [2023-10-12 23:39:08,015][44958] Updated weights for policy 0, policy_version 85720 (0.0008) [2023-10-12 23:39:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 175996928. Throughput: 0: 1630.1, 1: 1657.7. Samples: 44011362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:11,443][43579] Avg episode reward: [(0, '273.640'), (1, '289.470')] [2023-10-12 23:39:11,811][44959] Updated weights for policy 1, policy_version 86150 (0.0009) [2023-10-12 23:39:12,064][44958] Updated weights for policy 0, policy_version 85730 (0.0011) [2023-10-12 23:39:12,184][44959] Updated weights for policy 1, policy_version 86160 (0.0007) [2023-10-12 23:39:12,447][44958] Updated weights for policy 0, policy_version 85740 (0.0009) [2023-10-12 23:39:12,554][44959] Updated weights for policy 1, policy_version 86170 (0.0008) [2023-10-12 23:39:12,809][44958] Updated weights for policy 0, policy_version 85750 (0.0008) [2023-10-12 23:39:13,187][44958] Updated weights for policy 0, policy_version 85760 (0.0009) [2023-10-12 23:39:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176062464. Throughput: 0: 1635.8, 1: 1653.5. Samples: 44031936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:16,443][43579] Avg episode reward: [(0, '271.650'), (1, '290.610')] [2023-10-12 23:39:16,681][44959] Updated weights for policy 1, policy_version 86180 (0.0007) [2023-10-12 23:39:17,041][44959] Updated weights for policy 1, policy_version 86190 (0.0007) [2023-10-12 23:39:17,349][44958] Updated weights for policy 0, policy_version 85770 (0.0009) [2023-10-12 23:39:17,413][44959] Updated weights for policy 1, policy_version 86200 (0.0008) [2023-10-12 23:39:17,713][44958] Updated weights for policy 0, policy_version 85780 (0.0008) [2023-10-12 23:39:18,084][44958] Updated weights for policy 0, policy_version 85790 (0.0009) [2023-10-12 23:39:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176128000. Throughput: 0: 1633.0, 1: 1658.3. Samples: 44040672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:21,444][43579] Avg episode reward: [(0, '275.630'), (1, '291.280')] [2023-10-12 23:39:21,545][44959] Updated weights for policy 1, policy_version 86210 (0.0009) [2023-10-12 23:39:21,944][44959] Updated weights for policy 1, policy_version 86220 (0.0008) [2023-10-12 23:39:22,317][44959] Updated weights for policy 1, policy_version 86230 (0.0007) [2023-10-12 23:39:22,362][44958] Updated weights for policy 0, policy_version 85800 (0.0008) [2023-10-12 23:39:22,689][44959] Updated weights for policy 1, policy_version 86240 (0.0009) [2023-10-12 23:39:22,729][44958] Updated weights for policy 0, policy_version 85810 (0.0009) [2023-10-12 23:39:23,095][44958] Updated weights for policy 0, policy_version 85820 (0.0009) [2023-10-12 23:39:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176193536. Throughput: 0: 1636.8, 1: 1657.5. Samples: 44060756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:26,443][43579] Avg episode reward: [(0, '277.530'), (1, '292.080')] [2023-10-12 23:39:26,912][44959] Updated weights for policy 1, policy_version 86250 (0.0010) [2023-10-12 23:39:27,200][44958] Updated weights for policy 0, policy_version 85830 (0.0009) [2023-10-12 23:39:27,283][44959] Updated weights for policy 1, policy_version 86260 (0.0008) [2023-10-12 23:39:27,566][44958] Updated weights for policy 0, policy_version 85840 (0.0008) [2023-10-12 23:39:27,654][44959] Updated weights for policy 1, policy_version 86270 (0.0009) [2023-10-12 23:39:27,946][44958] Updated weights for policy 0, policy_version 85850 (0.0008) [2023-10-12 23:39:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176259072. Throughput: 0: 1640.6, 1: 1655.4. Samples: 44081006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:31,443][43579] Avg episode reward: [(0, '282.340'), (1, '288.780')] [2023-10-12 23:39:31,871][44959] Updated weights for policy 1, policy_version 86280 (0.0008) [2023-10-12 23:39:32,181][44958] Updated weights for policy 0, policy_version 85860 (0.0008) [2023-10-12 23:39:32,232][44959] Updated weights for policy 1, policy_version 86290 (0.0007) [2023-10-12 23:39:32,561][44958] Updated weights for policy 0, policy_version 85870 (0.0008) [2023-10-12 23:39:32,599][44959] Updated weights for policy 1, policy_version 86300 (0.0008) [2023-10-12 23:39:32,926][44958] Updated weights for policy 0, policy_version 85880 (0.0008) [2023-10-12 23:39:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176324608. Throughput: 0: 1641.7, 1: 1659.7. Samples: 44089920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:36,444][43579] Avg episode reward: [(0, '281.220'), (1, '288.390')] [2023-10-12 23:39:36,857][44958] Updated weights for policy 0, policy_version 85890 (0.0009) [2023-10-12 23:39:36,864][44959] Updated weights for policy 1, policy_version 86310 (0.0008) [2023-10-12 23:39:37,228][44958] Updated weights for policy 0, policy_version 85900 (0.0008) [2023-10-12 23:39:37,237][44959] Updated weights for policy 1, policy_version 86320 (0.0009) [2023-10-12 23:39:37,594][44958] Updated weights for policy 0, policy_version 85910 (0.0007) [2023-10-12 23:39:37,599][44959] Updated weights for policy 1, policy_version 86330 (0.0009) [2023-10-12 23:39:37,961][44958] Updated weights for policy 0, policy_version 85920 (0.0007) [2023-10-12 23:39:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176390144. Throughput: 0: 1643.6, 1: 1648.9. Samples: 44109930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:41,444][43579] Avg episode reward: [(0, '279.990'), (1, '287.710')] [2023-10-12 23:39:41,766][44959] Updated weights for policy 1, policy_version 86340 (0.0007) [2023-10-12 23:39:42,077][44958] Updated weights for policy 0, policy_version 85930 (0.0008) [2023-10-12 23:39:42,137][44959] Updated weights for policy 1, policy_version 86350 (0.0008) [2023-10-12 23:39:42,454][44958] Updated weights for policy 0, policy_version 85940 (0.0008) [2023-10-12 23:39:42,503][44959] Updated weights for policy 1, policy_version 86360 (0.0008) [2023-10-12 23:39:42,815][44958] Updated weights for policy 0, policy_version 85950 (0.0009) [2023-10-12 23:39:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176455680. Throughput: 0: 1648.3, 1: 1647.8. Samples: 44130452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:46,443][43579] Avg episode reward: [(0, '275.540'), (1, '286.030')] [2023-10-12 23:39:46,451][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000085952_88014848.pth... [2023-10-12 23:39:46,488][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000084448_86474752.pth [2023-10-12 23:39:46,607][44959] Updated weights for policy 1, policy_version 86370 (0.0008) [2023-10-12 23:39:46,975][44959] Updated weights for policy 1, policy_version 86380 (0.0008) [2023-10-12 23:39:47,099][44958] Updated weights for policy 0, policy_version 85960 (0.0008) [2023-10-12 23:39:47,340][44959] Updated weights for policy 1, policy_version 86390 (0.0009) [2023-10-12 23:39:47,477][44958] Updated weights for policy 0, policy_version 85970 (0.0007) [2023-10-12 23:39:47,707][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000086400_88473600.pth... [2023-10-12 23:39:47,713][44959] Updated weights for policy 1, policy_version 86400 (0.0009) [2023-10-12 23:39:47,737][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000084864_86900736.pth [2023-10-12 23:39:47,840][44958] Updated weights for policy 0, policy_version 85980 (0.0009) [2023-10-12 23:39:51,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176521216. Throughput: 0: 1647.2, 1: 1642.9. Samples: 44139052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:39:51,444][43579] Avg episode reward: [(0, '281.320'), (1, '285.640')] [2023-10-12 23:39:52,068][44959] Updated weights for policy 1, policy_version 86410 (0.0007) [2023-10-12 23:39:52,089][44958] Updated weights for policy 0, policy_version 85990 (0.0008) [2023-10-12 23:39:52,442][44959] Updated weights for policy 1, policy_version 86420 (0.0008) [2023-10-12 23:39:52,464][44958] Updated weights for policy 0, policy_version 86000 (0.0007) [2023-10-12 23:39:52,812][44959] Updated weights for policy 1, policy_version 86430 (0.0008) [2023-10-12 23:39:52,834][44958] Updated weights for policy 0, policy_version 86010 (0.0009) [2023-10-12 23:39:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176586752. Throughput: 0: 1650.4, 1: 1633.6. Samples: 44159138. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:39:56,443][43579] Avg episode reward: [(0, '280.320'), (1, '282.830')] [2023-10-12 23:39:56,833][44958] Updated weights for policy 0, policy_version 86020 (0.0007) [2023-10-12 23:39:57,064][44959] Updated weights for policy 1, policy_version 86440 (0.0009) [2023-10-12 23:39:57,199][44958] Updated weights for policy 0, policy_version 86030 (0.0009) [2023-10-12 23:39:57,433][44959] Updated weights for policy 1, policy_version 86450 (0.0007) [2023-10-12 23:39:57,567][44958] Updated weights for policy 0, policy_version 86040 (0.0007) [2023-10-12 23:39:57,792][44959] Updated weights for policy 1, policy_version 86460 (0.0009) [2023-10-12 23:40:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 176652288. Throughput: 0: 1640.5, 1: 1629.3. Samples: 44179078. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:01,444][43579] Avg episode reward: [(0, '279.710'), (1, '286.700')] [2023-10-12 23:40:01,811][44958] Updated weights for policy 0, policy_version 86050 (0.0007) [2023-10-12 23:40:01,883][44959] Updated weights for policy 1, policy_version 86470 (0.0010) [2023-10-12 23:40:02,182][44958] Updated weights for policy 0, policy_version 86060 (0.0007) [2023-10-12 23:40:02,250][44959] Updated weights for policy 1, policy_version 86480 (0.0009) [2023-10-12 23:40:02,554][44958] Updated weights for policy 0, policy_version 86070 (0.0007) [2023-10-12 23:40:02,630][44959] Updated weights for policy 1, policy_version 86490 (0.0007) [2023-10-12 23:40:02,921][44958] Updated weights for policy 0, policy_version 86080 (0.0007) [2023-10-12 23:40:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176717824. Throughput: 0: 1644.6, 1: 1624.8. Samples: 44187794. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:06,444][43579] Avg episode reward: [(0, '280.890'), (1, '288.300')] [2023-10-12 23:40:06,892][44959] Updated weights for policy 1, policy_version 86500 (0.0007) [2023-10-12 23:40:07,109][44958] Updated weights for policy 0, policy_version 86090 (0.0008) [2023-10-12 23:40:07,282][44959] Updated weights for policy 1, policy_version 86510 (0.0008) [2023-10-12 23:40:07,480][44958] Updated weights for policy 0, policy_version 86100 (0.0010) [2023-10-12 23:40:07,656][44959] Updated weights for policy 1, policy_version 86520 (0.0008) [2023-10-12 23:40:07,839][44958] Updated weights for policy 0, policy_version 86110 (0.0007) [2023-10-12 23:40:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176783360. Throughput: 0: 1643.4, 1: 1622.2. Samples: 44207710. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:11,443][43579] Avg episode reward: [(0, '285.340'), (1, '287.760')] [2023-10-12 23:40:11,891][44959] Updated weights for policy 1, policy_version 86530 (0.0007) [2023-10-12 23:40:12,158][44958] Updated weights for policy 0, policy_version 86120 (0.0008) [2023-10-12 23:40:12,259][44959] Updated weights for policy 1, policy_version 86540 (0.0008) [2023-10-12 23:40:12,525][44958] Updated weights for policy 0, policy_version 86130 (0.0010) [2023-10-12 23:40:12,626][44959] Updated weights for policy 1, policy_version 86550 (0.0008) [2023-10-12 23:40:12,894][44958] Updated weights for policy 0, policy_version 86140 (0.0007) [2023-10-12 23:40:12,993][44959] Updated weights for policy 1, policy_version 86560 (0.0008) [2023-10-12 23:40:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176848896. Throughput: 0: 1639.5, 1: 1628.2. Samples: 44228052. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:16,443][43579] Avg episode reward: [(0, '285.880'), (1, '290.530')] [2023-10-12 23:40:17,019][44958] Updated weights for policy 0, policy_version 86150 (0.0008) [2023-10-12 23:40:17,188][44959] Updated weights for policy 1, policy_version 86570 (0.0008) [2023-10-12 23:40:17,403][44958] Updated weights for policy 0, policy_version 86160 (0.0007) [2023-10-12 23:40:17,559][44959] Updated weights for policy 1, policy_version 86580 (0.0009) [2023-10-12 23:40:17,771][44958] Updated weights for policy 0, policy_version 86170 (0.0008) [2023-10-12 23:40:17,923][44959] Updated weights for policy 1, policy_version 86590 (0.0008) [2023-10-12 23:40:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176914432. Throughput: 0: 1636.7, 1: 1626.6. Samples: 44236770. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:21,444][43579] Avg episode reward: [(0, '281.450'), (1, '291.840')] [2023-10-12 23:40:22,052][44959] Updated weights for policy 1, policy_version 86600 (0.0007) [2023-10-12 23:40:22,071][44958] Updated weights for policy 0, policy_version 86180 (0.0010) [2023-10-12 23:40:22,424][44959] Updated weights for policy 1, policy_version 86610 (0.0007) [2023-10-12 23:40:22,440][44958] Updated weights for policy 0, policy_version 86190 (0.0010) [2023-10-12 23:40:22,782][44959] Updated weights for policy 1, policy_version 86620 (0.0008) [2023-10-12 23:40:22,814][44958] Updated weights for policy 0, policy_version 86200 (0.0008) [2023-10-12 23:40:26,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 176979968. Throughput: 0: 1633.3, 1: 1632.9. Samples: 44256904. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:26,443][43579] Avg episode reward: [(0, '276.730'), (1, '294.050')] [2023-10-12 23:40:26,444][44583] Saving new best policy, reward=294.050! [2023-10-12 23:40:26,885][44958] Updated weights for policy 0, policy_version 86210 (0.0011) [2023-10-12 23:40:26,974][44959] Updated weights for policy 1, policy_version 86630 (0.0008) [2023-10-12 23:40:27,245][44958] Updated weights for policy 0, policy_version 86220 (0.0008) [2023-10-12 23:40:27,346][44959] Updated weights for policy 1, policy_version 86640 (0.0007) [2023-10-12 23:40:27,622][44958] Updated weights for policy 0, policy_version 86230 (0.0009) [2023-10-12 23:40:27,718][44959] Updated weights for policy 1, policy_version 86650 (0.0007) [2023-10-12 23:40:27,987][44958] Updated weights for policy 0, policy_version 86240 (0.0009) [2023-10-12 23:40:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 177045504. Throughput: 0: 1631.4, 1: 1627.6. Samples: 44277106. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:31,443][43579] Avg episode reward: [(0, '277.210'), (1, '287.460')] [2023-10-12 23:40:31,981][44959] Updated weights for policy 1, policy_version 86660 (0.0010) [2023-10-12 23:40:32,163][44958] Updated weights for policy 0, policy_version 86250 (0.0008) [2023-10-12 23:40:32,354][44959] Updated weights for policy 1, policy_version 86670 (0.0009) [2023-10-12 23:40:32,537][44958] Updated weights for policy 0, policy_version 86260 (0.0008) [2023-10-12 23:40:32,719][44959] Updated weights for policy 1, policy_version 86680 (0.0009) [2023-10-12 23:40:32,912][44958] Updated weights for policy 0, policy_version 86270 (0.0008) [2023-10-12 23:40:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 177111040. Throughput: 0: 1634.2, 1: 1629.5. Samples: 44285918. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:36,443][43579] Avg episode reward: [(0, '271.640'), (1, '282.860')] [2023-10-12 23:40:36,802][44959] Updated weights for policy 1, policy_version 86690 (0.0008) [2023-10-12 23:40:37,047][44958] Updated weights for policy 0, policy_version 86280 (0.0008) [2023-10-12 23:40:37,172][44959] Updated weights for policy 1, policy_version 86700 (0.0008) [2023-10-12 23:40:37,416][44958] Updated weights for policy 0, policy_version 86290 (0.0007) [2023-10-12 23:40:37,535][44959] Updated weights for policy 1, policy_version 86710 (0.0007) [2023-10-12 23:40:37,782][44958] Updated weights for policy 0, policy_version 86300 (0.0008) [2023-10-12 23:40:37,901][44959] Updated weights for policy 1, policy_version 86720 (0.0007) [2023-10-12 23:40:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 177176576. Throughput: 0: 1634.4, 1: 1637.4. Samples: 44306372. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:41,444][43579] Avg episode reward: [(0, '270.830'), (1, '277.170')] [2023-10-12 23:40:41,939][44958] Updated weights for policy 0, policy_version 86310 (0.0008) [2023-10-12 23:40:42,008][44959] Updated weights for policy 1, policy_version 86730 (0.0009) [2023-10-12 23:40:42,306][44958] Updated weights for policy 0, policy_version 86320 (0.0009) [2023-10-12 23:40:42,376][44959] Updated weights for policy 1, policy_version 86740 (0.0008) [2023-10-12 23:40:42,671][44958] Updated weights for policy 0, policy_version 86330 (0.0009) [2023-10-12 23:40:42,742][44959] Updated weights for policy 1, policy_version 86750 (0.0009) [2023-10-12 23:40:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 177242112. Throughput: 0: 1634.9, 1: 1635.7. Samples: 44326254. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-12 23:40:46,443][43579] Avg episode reward: [(0, '274.140'), (1, '275.780')] [2023-10-12 23:40:47,046][44959] Updated weights for policy 1, policy_version 86760 (0.0008) [2023-10-12 23:40:47,169][44958] Updated weights for policy 0, policy_version 86340 (0.0009) [2023-10-12 23:40:47,413][44959] Updated weights for policy 1, policy_version 86770 (0.0008) [2023-10-12 23:40:47,540][44958] Updated weights for policy 0, policy_version 86350 (0.0009) [2023-10-12 23:40:47,772][44959] Updated weights for policy 1, policy_version 86780 (0.0009) [2023-10-12 23:40:47,912][44958] Updated weights for policy 0, policy_version 86360 (0.0010) [2023-10-12 23:40:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 177307648. Throughput: 0: 1635.3, 1: 1636.2. Samples: 44335012. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:40:51,444][43579] Avg episode reward: [(0, '276.040'), (1, '271.650')] [2023-10-12 23:40:51,925][44959] Updated weights for policy 1, policy_version 86790 (0.0009) [2023-10-12 23:40:52,229][44958] Updated weights for policy 0, policy_version 86370 (0.0009) [2023-10-12 23:40:52,284][44959] Updated weights for policy 1, policy_version 86800 (0.0007) [2023-10-12 23:40:52,619][44958] Updated weights for policy 0, policy_version 86380 (0.0008) [2023-10-12 23:40:52,651][44959] Updated weights for policy 1, policy_version 86810 (0.0007) [2023-10-12 23:40:52,996][44958] Updated weights for policy 0, policy_version 86390 (0.0010) [2023-10-12 23:40:53,358][44958] Updated weights for policy 0, policy_version 86400 (0.0010) [2023-10-12 23:40:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.1). Total num frames: 177373184. Throughput: 0: 1638.1, 1: 1642.1. Samples: 44355322. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:40:56,443][43579] Avg episode reward: [(0, '278.080'), (1, '275.170')] [2023-10-12 23:40:56,702][44959] Updated weights for policy 1, policy_version 86820 (0.0008) [2023-10-12 23:40:57,083][44959] Updated weights for policy 1, policy_version 86830 (0.0010) [2023-10-12 23:40:57,448][44959] Updated weights for policy 1, policy_version 86840 (0.0008) [2023-10-12 23:40:57,463][44958] Updated weights for policy 0, policy_version 86410 (0.0009) [2023-10-12 23:40:57,836][44958] Updated weights for policy 0, policy_version 86420 (0.0009) [2023-10-12 23:40:58,209][44958] Updated weights for policy 0, policy_version 86430 (0.0008) [2023-10-12 23:41:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 177438720. Throughput: 0: 1637.3, 1: 1640.4. Samples: 44375552. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:41:01,444][43579] Avg episode reward: [(0, '276.140'), (1, '275.420')] [2023-10-12 23:41:01,677][44959] Updated weights for policy 1, policy_version 86850 (0.0007) [2023-10-12 23:41:02,053][44959] Updated weights for policy 1, policy_version 86860 (0.0008) [2023-10-12 23:41:02,245][44958] Updated weights for policy 0, policy_version 86440 (0.0008) [2023-10-12 23:41:02,418][44959] Updated weights for policy 1, policy_version 86870 (0.0007) [2023-10-12 23:41:02,621][44958] Updated weights for policy 0, policy_version 86450 (0.0009) [2023-10-12 23:41:02,785][44959] Updated weights for policy 1, policy_version 86880 (0.0007) [2023-10-12 23:41:02,995][44958] Updated weights for policy 0, policy_version 86460 (0.0007) [2023-10-12 23:41:06,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 177504256. Throughput: 0: 1643.7, 1: 1640.4. Samples: 44384556. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:41:06,444][43579] Avg episode reward: [(0, '280.060'), (1, '279.850')] [2023-10-12 23:41:06,869][44959] Updated weights for policy 1, policy_version 86890 (0.0007) [2023-10-12 23:41:07,193][44958] Updated weights for policy 0, policy_version 86470 (0.0008) [2023-10-12 23:41:07,240][44959] Updated weights for policy 1, policy_version 86900 (0.0009) [2023-10-12 23:41:07,561][44958] Updated weights for policy 0, policy_version 86480 (0.0010) [2023-10-12 23:41:07,610][44959] Updated weights for policy 1, policy_version 86910 (0.0009) [2023-10-12 23:41:07,928][44958] Updated weights for policy 0, policy_version 86490 (0.0009) [2023-10-12 23:41:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 177569792. Throughput: 0: 1639.8, 1: 1644.8. Samples: 44404714. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:41:11,443][43579] Avg episode reward: [(0, '277.230'), (1, '279.870')] [2023-10-12 23:41:11,779][44959] Updated weights for policy 1, policy_version 86920 (0.0008) [2023-10-12 23:41:12,133][44958] Updated weights for policy 0, policy_version 86500 (0.0009) [2023-10-12 23:41:12,149][44959] Updated weights for policy 1, policy_version 86930 (0.0007) [2023-10-12 23:41:12,505][44958] Updated weights for policy 0, policy_version 86510 (0.0010) [2023-10-12 23:41:12,517][44959] Updated weights for policy 1, policy_version 86940 (0.0007) [2023-10-12 23:41:12,891][44958] Updated weights for policy 0, policy_version 86520 (0.0008) [2023-10-12 23:41:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 177635328. Throughput: 0: 1644.4, 1: 1650.5. Samples: 44425376. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:41:16,443][43579] Avg episode reward: [(0, '275.740'), (1, '281.490')] [2023-10-12 23:41:16,592][44959] Updated weights for policy 1, policy_version 86950 (0.0009) [2023-10-12 23:41:16,733][44958] Updated weights for policy 0, policy_version 86530 (0.0008) [2023-10-12 23:41:16,953][44959] Updated weights for policy 1, policy_version 86960 (0.0007) [2023-10-12 23:41:17,100][44958] Updated weights for policy 0, policy_version 86540 (0.0008) [2023-10-12 23:41:17,319][44959] Updated weights for policy 1, policy_version 86970 (0.0007) [2023-10-12 23:41:17,482][44958] Updated weights for policy 0, policy_version 86550 (0.0009) [2023-10-12 23:41:17,861][44958] Updated weights for policy 0, policy_version 86560 (0.0009) [2023-10-12 23:41:21,392][44959] Updated weights for policy 1, policy_version 86980 (0.0008) [2023-10-12 23:41:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 177700864. Throughput: 0: 1645.1, 1: 1649.8. Samples: 44434186. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:41:21,443][43579] Avg episode reward: [(0, '274.620'), (1, '282.730')] [2023-10-12 23:41:21,760][44959] Updated weights for policy 1, policy_version 86990 (0.0007) [2023-10-12 23:41:22,132][44959] Updated weights for policy 1, policy_version 87000 (0.0009) [2023-10-12 23:41:22,136][44958] Updated weights for policy 0, policy_version 86570 (0.0009) [2023-10-12 23:41:22,508][44958] Updated weights for policy 0, policy_version 86580 (0.0009) [2023-10-12 23:41:22,875][44958] Updated weights for policy 0, policy_version 86590 (0.0009) [2023-10-12 23:41:26,340][44959] Updated weights for policy 1, policy_version 87010 (0.0007) [2023-10-12 23:41:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 177766400. Throughput: 0: 1638.2, 1: 1652.2. Samples: 44454440. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:41:26,443][43579] Avg episode reward: [(0, '276.180'), (1, '286.550')] [2023-10-12 23:41:26,696][44959] Updated weights for policy 1, policy_version 87020 (0.0009) [2023-10-12 23:41:27,066][44959] Updated weights for policy 1, policy_version 87030 (0.0009) [2023-10-12 23:41:27,240][44958] Updated weights for policy 0, policy_version 86600 (0.0009) [2023-10-12 23:41:27,421][44959] Updated weights for policy 1, policy_version 87040 (0.0008) [2023-10-12 23:41:27,614][44958] Updated weights for policy 0, policy_version 86610 (0.0010) [2023-10-12 23:41:27,982][44958] Updated weights for policy 0, policy_version 86620 (0.0009) [2023-10-12 23:41:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 177831936. Throughput: 0: 1645.1, 1: 1655.8. Samples: 44474794. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:41:31,443][43579] Avg episode reward: [(0, '277.290'), (1, '286.860')] [2023-10-12 23:41:31,546][44959] Updated weights for policy 1, policy_version 87050 (0.0010) [2023-10-12 23:41:31,880][44958] Updated weights for policy 0, policy_version 86630 (0.0010) [2023-10-12 23:41:31,924][44959] Updated weights for policy 1, policy_version 87060 (0.0008) [2023-10-12 23:41:32,254][44958] Updated weights for policy 0, policy_version 86640 (0.0008) [2023-10-12 23:41:32,294][44959] Updated weights for policy 1, policy_version 87070 (0.0008) [2023-10-12 23:41:32,617][44958] Updated weights for policy 0, policy_version 86650 (0.0008) [2023-10-12 23:41:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 177897472. Throughput: 0: 1646.0, 1: 1656.0. Samples: 44483602. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:41:36,443][43579] Avg episode reward: [(0, '278.840'), (1, '283.030')] [2023-10-12 23:41:36,634][44959] Updated weights for policy 1, policy_version 87080 (0.0009) [2023-10-12 23:41:36,778][44958] Updated weights for policy 0, policy_version 86660 (0.0010) [2023-10-12 23:41:37,005][44959] Updated weights for policy 1, policy_version 87090 (0.0008) [2023-10-12 23:41:37,143][44958] Updated weights for policy 0, policy_version 86670 (0.0007) [2023-10-12 23:41:37,368][44959] Updated weights for policy 1, policy_version 87100 (0.0009) [2023-10-12 23:41:37,518][44958] Updated weights for policy 0, policy_version 86680 (0.0008) [2023-10-12 23:41:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.1). Total num frames: 177963008. Throughput: 0: 1644.8, 1: 1653.6. Samples: 44503754. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-12 23:41:41,443][43579] Avg episode reward: [(0, '277.210'), (1, '281.550')] [2023-10-12 23:41:41,558][44959] Updated weights for policy 1, policy_version 87110 (0.0009) [2023-10-12 23:41:41,681][44958] Updated weights for policy 0, policy_version 86690 (0.0009) [2023-10-12 23:41:41,926][44959] Updated weights for policy 1, policy_version 87120 (0.0008) [2023-10-12 23:41:42,074][44958] Updated weights for policy 0, policy_version 86700 (0.0008) [2023-10-12 23:41:42,301][44959] Updated weights for policy 1, policy_version 87130 (0.0009) [2023-10-12 23:41:42,450][44958] Updated weights for policy 0, policy_version 86710 (0.0008) [2023-10-12 23:41:42,832][44958] Updated weights for policy 0, policy_version 86720 (0.0010) [2023-10-12 23:41:46,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 12885.0). Total num frames: 178028544. Throughput: 0: 1648.4, 1: 1644.9. Samples: 44523752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:41:46,444][43579] Avg episode reward: [(0, '275.830'), (1, '283.010')] [2023-10-12 23:41:46,455][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000087136_89227264.pth... [2023-10-12 23:41:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000085632_87687168.pth [2023-10-12 23:41:46,749][44959] Updated weights for policy 1, policy_version 87140 (0.0008) [2023-10-12 23:41:46,910][44958] Updated weights for policy 0, policy_version 86730 (0.0009) [2023-10-12 23:41:47,124][44959] Updated weights for policy 1, policy_version 87150 (0.0008) [2023-10-12 23:41:47,283][44958] Updated weights for policy 0, policy_version 86740 (0.0008) [2023-10-12 23:41:47,494][44959] Updated weights for policy 1, policy_version 87160 (0.0008) [2023-10-12 23:41:47,659][44958] Updated weights for policy 0, policy_version 86750 (0.0009) [2023-10-12 23:41:47,726][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000086752_88834048.pth... [2023-10-12 23:41:47,757][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000085216_87261184.pth [2023-10-12 23:41:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 178094080. Throughput: 0: 1641.2, 1: 1644.1. Samples: 44532396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:41:51,443][43579] Avg episode reward: [(0, '283.120'), (1, '282.670')] [2023-10-12 23:41:51,643][44959] Updated weights for policy 1, policy_version 87170 (0.0007) [2023-10-12 23:41:51,864][44958] Updated weights for policy 0, policy_version 86760 (0.0009) [2023-10-12 23:41:52,007][44959] Updated weights for policy 1, policy_version 87180 (0.0008) [2023-10-12 23:41:52,244][44958] Updated weights for policy 0, policy_version 86770 (0.0009) [2023-10-12 23:41:52,364][44959] Updated weights for policy 1, policy_version 87190 (0.0007) [2023-10-12 23:41:52,618][44958] Updated weights for policy 0, policy_version 86780 (0.0008) [2023-10-12 23:41:52,732][44959] Updated weights for policy 1, policy_version 87200 (0.0007) [2023-10-12 23:41:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 178159616. Throughput: 0: 1650.5, 1: 1636.8. Samples: 44552644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:41:56,443][43579] Avg episode reward: [(0, '282.970'), (1, '277.430')] [2023-10-12 23:41:56,692][44958] Updated weights for policy 0, policy_version 86790 (0.0010) [2023-10-12 23:41:57,067][44958] Updated weights for policy 0, policy_version 86800 (0.0009) [2023-10-12 23:41:57,118][44959] Updated weights for policy 1, policy_version 87210 (0.0008) [2023-10-12 23:41:57,435][44958] Updated weights for policy 0, policy_version 86810 (0.0008) [2023-10-12 23:41:57,489][44959] Updated weights for policy 1, policy_version 87220 (0.0008) [2023-10-12 23:41:57,850][44959] Updated weights for policy 1, policy_version 87230 (0.0009) [2023-10-12 23:42:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 178225152. Throughput: 0: 1640.8, 1: 1633.4. Samples: 44572716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:42:01,444][43579] Avg episode reward: [(0, '279.770'), (1, '278.570')] [2023-10-12 23:42:01,729][44958] Updated weights for policy 0, policy_version 86820 (0.0007) [2023-10-12 23:42:01,933][44959] Updated weights for policy 1, policy_version 87240 (0.0009) [2023-10-12 23:42:02,096][44958] Updated weights for policy 0, policy_version 86830 (0.0009) [2023-10-12 23:42:02,303][44959] Updated weights for policy 1, policy_version 87250 (0.0008) [2023-10-12 23:42:02,468][44958] Updated weights for policy 0, policy_version 86840 (0.0008) [2023-10-12 23:42:02,665][44959] Updated weights for policy 1, policy_version 87260 (0.0009) [2023-10-12 23:42:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 178290688. Throughput: 0: 1640.0, 1: 1632.8. Samples: 44581462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:42:06,443][43579] Avg episode reward: [(0, '277.640'), (1, '282.870')] [2023-10-12 23:42:06,589][44959] Updated weights for policy 1, policy_version 87270 (0.0008) [2023-10-12 23:42:06,705][44958] Updated weights for policy 0, policy_version 86850 (0.0008) [2023-10-12 23:42:06,961][44959] Updated weights for policy 1, policy_version 87280 (0.0009) [2023-10-12 23:42:07,076][44958] Updated weights for policy 0, policy_version 86860 (0.0009) [2023-10-12 23:42:07,327][44959] Updated weights for policy 1, policy_version 87290 (0.0007) [2023-10-12 23:42:07,452][44958] Updated weights for policy 0, policy_version 86870 (0.0008) [2023-10-12 23:42:07,828][44958] Updated weights for policy 0, policy_version 86880 (0.0009) [2023-10-12 23:42:11,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12885.1). Total num frames: 178356224. Throughput: 0: 1645.0, 1: 1635.4. Samples: 44602056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:42:11,443][43579] Avg episode reward: [(0, '278.510'), (1, '282.540')] [2023-10-12 23:42:11,606][44959] Updated weights for policy 1, policy_version 87300 (0.0008) [2023-10-12 23:42:11,977][44959] Updated weights for policy 1, policy_version 87310 (0.0008) [2023-10-12 23:42:12,039][44958] Updated weights for policy 0, policy_version 86890 (0.0009) [2023-10-12 23:42:12,342][44959] Updated weights for policy 1, policy_version 87320 (0.0010) [2023-10-12 23:42:12,406][44958] Updated weights for policy 0, policy_version 86900 (0.0009) [2023-10-12 23:42:12,776][44958] Updated weights for policy 0, policy_version 86910 (0.0008) [2023-10-12 23:42:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 178421760. Throughput: 0: 1640.5, 1: 1633.8. Samples: 44622138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:42:16,444][43579] Avg episode reward: [(0, '279.420'), (1, '277.910')] [2023-10-12 23:42:16,505][44959] Updated weights for policy 1, policy_version 87330 (0.0008) [2023-10-12 23:42:16,874][44959] Updated weights for policy 1, policy_version 87340 (0.0010) [2023-10-12 23:42:17,017][44958] Updated weights for policy 0, policy_version 86920 (0.0008) [2023-10-12 23:42:17,244][44959] Updated weights for policy 1, policy_version 87350 (0.0007) [2023-10-12 23:42:17,392][44958] Updated weights for policy 0, policy_version 86930 (0.0008) [2023-10-12 23:42:17,618][44959] Updated weights for policy 1, policy_version 87360 (0.0007) [2023-10-12 23:42:17,771][44958] Updated weights for policy 0, policy_version 86940 (0.0009) [2023-10-12 23:42:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 178487296. Throughput: 0: 1639.6, 1: 1635.7. Samples: 44630992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:42:21,443][43579] Avg episode reward: [(0, '276.210'), (1, '276.740')] [2023-10-12 23:42:21,785][44959] Updated weights for policy 1, policy_version 87370 (0.0011) [2023-10-12 23:42:21,922][44958] Updated weights for policy 0, policy_version 86950 (0.0009) [2023-10-12 23:42:22,160][44959] Updated weights for policy 1, policy_version 87380 (0.0009) [2023-10-12 23:42:22,291][44958] Updated weights for policy 0, policy_version 86960 (0.0008) [2023-10-12 23:42:22,526][44959] Updated weights for policy 1, policy_version 87390 (0.0008) [2023-10-12 23:42:22,664][44958] Updated weights for policy 0, policy_version 86970 (0.0009) [2023-10-12 23:42:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 178552832. Throughput: 0: 1639.5, 1: 1633.2. Samples: 44651024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:42:26,444][43579] Avg episode reward: [(0, '280.130'), (1, '283.400')] [2023-10-12 23:42:26,751][44959] Updated weights for policy 1, policy_version 87400 (0.0010) [2023-10-12 23:42:26,922][44958] Updated weights for policy 0, policy_version 86980 (0.0009) [2023-10-12 23:42:27,123][44959] Updated weights for policy 1, policy_version 87410 (0.0010) [2023-10-12 23:42:27,319][44958] Updated weights for policy 0, policy_version 86990 (0.0009) [2023-10-12 23:42:27,487][44959] Updated weights for policy 1, policy_version 87420 (0.0009) [2023-10-12 23:42:27,686][44958] Updated weights for policy 0, policy_version 87000 (0.0011) [2023-10-12 23:42:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 178618368. Throughput: 0: 1635.0, 1: 1639.8. Samples: 44671116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:42:31,443][43579] Avg episode reward: [(0, '280.130'), (1, '283.510')] [2023-10-12 23:42:31,704][44959] Updated weights for policy 1, policy_version 87430 (0.0009) [2023-10-12 23:42:31,797][44958] Updated weights for policy 0, policy_version 87010 (0.0009) [2023-10-12 23:42:32,071][44959] Updated weights for policy 1, policy_version 87440 (0.0009) [2023-10-12 23:42:32,159][44958] Updated weights for policy 0, policy_version 87020 (0.0010) [2023-10-12 23:42:32,441][44959] Updated weights for policy 1, policy_version 87450 (0.0008) [2023-10-12 23:42:32,529][44958] Updated weights for policy 0, policy_version 87030 (0.0008) [2023-10-12 23:42:32,894][44958] Updated weights for policy 0, policy_version 87040 (0.0010) [2023-10-12 23:42:36,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 178683904. Throughput: 0: 1639.5, 1: 1640.9. Samples: 44680014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:42:36,443][43579] Avg episode reward: [(0, '282.330'), (1, '282.340')] [2023-10-12 23:42:36,528][44959] Updated weights for policy 1, policy_version 87460 (0.0009) [2023-10-12 23:42:36,902][44959] Updated weights for policy 1, policy_version 87470 (0.0010) [2023-10-12 23:42:36,976][44958] Updated weights for policy 0, policy_version 87050 (0.0007) [2023-10-12 23:42:37,268][44959] Updated weights for policy 1, policy_version 87480 (0.0009) [2023-10-12 23:42:37,351][44958] Updated weights for policy 0, policy_version 87060 (0.0007) [2023-10-12 23:42:37,729][44958] Updated weights for policy 0, policy_version 87070 (0.0009) [2023-10-12 23:42:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178749440. Throughput: 0: 1639.2, 1: 1646.4. Samples: 44700496. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:42:41,443][43579] Avg episode reward: [(0, '280.590'), (1, '284.550')] [2023-10-12 23:42:41,481][44959] Updated weights for policy 1, policy_version 87490 (0.0009) [2023-10-12 23:42:41,810][44958] Updated weights for policy 0, policy_version 87080 (0.0008) [2023-10-12 23:42:41,854][44959] Updated weights for policy 1, policy_version 87500 (0.0008) [2023-10-12 23:42:42,185][44958] Updated weights for policy 0, policy_version 87090 (0.0007) [2023-10-12 23:42:42,223][44959] Updated weights for policy 1, policy_version 87510 (0.0008) [2023-10-12 23:42:42,557][44958] Updated weights for policy 0, policy_version 87100 (0.0007) [2023-10-12 23:42:42,596][44959] Updated weights for policy 1, policy_version 87520 (0.0007) [2023-10-12 23:42:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178814976. Throughput: 0: 1635.6, 1: 1650.4. Samples: 44720586. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:42:46,444][43579] Avg episode reward: [(0, '283.800'), (1, '286.810')] [2023-10-12 23:42:46,637][44959] Updated weights for policy 1, policy_version 87530 (0.0009) [2023-10-12 23:42:46,929][44958] Updated weights for policy 0, policy_version 87110 (0.0009) [2023-10-12 23:42:47,002][44959] Updated weights for policy 1, policy_version 87540 (0.0008) [2023-10-12 23:42:47,301][44958] Updated weights for policy 0, policy_version 87120 (0.0008) [2023-10-12 23:42:47,380][44959] Updated weights for policy 1, policy_version 87550 (0.0010) [2023-10-12 23:42:47,680][44958] Updated weights for policy 0, policy_version 87130 (0.0010) [2023-10-12 23:42:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178880512. Throughput: 0: 1632.8, 1: 1651.3. Samples: 44729246. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:42:51,443][43579] Avg episode reward: [(0, '284.240'), (1, '288.240')] [2023-10-12 23:42:51,551][44959] Updated weights for policy 1, policy_version 87560 (0.0008) [2023-10-12 23:42:51,924][44959] Updated weights for policy 1, policy_version 87570 (0.0008) [2023-10-12 23:42:51,962][44958] Updated weights for policy 0, policy_version 87140 (0.0010) [2023-10-12 23:42:52,297][44959] Updated weights for policy 1, policy_version 87580 (0.0008) [2023-10-12 23:42:52,336][44958] Updated weights for policy 0, policy_version 87150 (0.0007) [2023-10-12 23:42:52,704][44958] Updated weights for policy 0, policy_version 87160 (0.0007) [2023-10-12 23:42:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 178946048. Throughput: 0: 1629.4, 1: 1642.8. Samples: 44749306. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:42:56,443][43579] Avg episode reward: [(0, '277.920'), (1, '288.770')] [2023-10-12 23:42:56,604][44959] Updated weights for policy 1, policy_version 87590 (0.0008) [2023-10-12 23:42:56,715][44958] Updated weights for policy 0, policy_version 87170 (0.0010) [2023-10-12 23:42:56,974][44959] Updated weights for policy 1, policy_version 87600 (0.0009) [2023-10-12 23:42:57,089][44958] Updated weights for policy 0, policy_version 87180 (0.0010) [2023-10-12 23:42:57,338][44959] Updated weights for policy 1, policy_version 87610 (0.0010) [2023-10-12 23:42:57,455][44958] Updated weights for policy 0, policy_version 87190 (0.0009) [2023-10-12 23:42:57,824][44958] Updated weights for policy 0, policy_version 87200 (0.0009) [2023-10-12 23:43:01,413][44959] Updated weights for policy 1, policy_version 87620 (0.0010) [2023-10-12 23:43:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179011584. Throughput: 0: 1631.7, 1: 1647.3. Samples: 44769696. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:43:01,443][43579] Avg episode reward: [(0, '279.120'), (1, '287.340')] [2023-10-12 23:43:01,779][44959] Updated weights for policy 1, policy_version 87630 (0.0008) [2023-10-12 23:43:02,087][44958] Updated weights for policy 0, policy_version 87210 (0.0007) [2023-10-12 23:43:02,143][44959] Updated weights for policy 1, policy_version 87640 (0.0009) [2023-10-12 23:43:02,451][44958] Updated weights for policy 0, policy_version 87220 (0.0008) [2023-10-12 23:43:02,821][44958] Updated weights for policy 0, policy_version 87230 (0.0008) [2023-10-12 23:43:06,177][44959] Updated weights for policy 1, policy_version 87650 (0.0008) [2023-10-12 23:43:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179077120. Throughput: 0: 1632.8, 1: 1646.6. Samples: 44778564. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:43:06,443][43579] Avg episode reward: [(0, '279.320'), (1, '286.670')] [2023-10-12 23:43:06,593][44959] Updated weights for policy 1, policy_version 87660 (0.0009) [2023-10-12 23:43:06,881][44958] Updated weights for policy 0, policy_version 87240 (0.0008) [2023-10-12 23:43:06,960][44959] Updated weights for policy 1, policy_version 87670 (0.0008) [2023-10-12 23:43:07,252][44958] Updated weights for policy 0, policy_version 87250 (0.0008) [2023-10-12 23:43:07,320][44959] Updated weights for policy 1, policy_version 87680 (0.0009) [2023-10-12 23:43:07,625][44958] Updated weights for policy 0, policy_version 87260 (0.0008) [2023-10-12 23:43:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179142656. Throughput: 0: 1639.3, 1: 1651.6. Samples: 44799116. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:43:11,443][43579] Avg episode reward: [(0, '274.500'), (1, '284.780')] [2023-10-12 23:43:11,463][44959] Updated weights for policy 1, policy_version 87690 (0.0008) [2023-10-12 23:43:11,777][44958] Updated weights for policy 0, policy_version 87270 (0.0008) [2023-10-12 23:43:11,837][44959] Updated weights for policy 1, policy_version 87700 (0.0009) [2023-10-12 23:43:12,161][44958] Updated weights for policy 0, policy_version 87280 (0.0009) [2023-10-12 23:43:12,196][44959] Updated weights for policy 1, policy_version 87710 (0.0009) [2023-10-12 23:43:12,525][44958] Updated weights for policy 0, policy_version 87290 (0.0008) [2023-10-12 23:43:16,316][44959] Updated weights for policy 1, policy_version 87720 (0.0008) [2023-10-12 23:43:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179208192. Throughput: 0: 1642.8, 1: 1649.0. Samples: 44819250. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:43:16,443][43579] Avg episode reward: [(0, '272.250'), (1, '286.170')] [2023-10-12 23:43:16,686][44959] Updated weights for policy 1, policy_version 87730 (0.0007) [2023-10-12 23:43:16,833][44958] Updated weights for policy 0, policy_version 87300 (0.0008) [2023-10-12 23:43:17,051][44959] Updated weights for policy 1, policy_version 87740 (0.0008) [2023-10-12 23:43:17,207][44958] Updated weights for policy 0, policy_version 87310 (0.0009) [2023-10-12 23:43:17,588][44958] Updated weights for policy 0, policy_version 87320 (0.0010) [2023-10-12 23:43:21,172][44959] Updated weights for policy 1, policy_version 87750 (0.0009) [2023-10-12 23:43:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179273728. Throughput: 0: 1642.8, 1: 1649.3. Samples: 44828160. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:43:21,443][43579] Avg episode reward: [(0, '272.840'), (1, '284.150')] [2023-10-12 23:43:21,537][44959] Updated weights for policy 1, policy_version 87760 (0.0010) [2023-10-12 23:43:21,799][44958] Updated weights for policy 0, policy_version 87330 (0.0010) [2023-10-12 23:43:21,912][44959] Updated weights for policy 1, policy_version 87770 (0.0009) [2023-10-12 23:43:22,182][44958] Updated weights for policy 0, policy_version 87340 (0.0008) [2023-10-12 23:43:22,558][44958] Updated weights for policy 0, policy_version 87350 (0.0009) [2023-10-12 23:43:22,928][44958] Updated weights for policy 0, policy_version 87360 (0.0011) [2023-10-12 23:43:26,119][44959] Updated weights for policy 1, policy_version 87780 (0.0009) [2023-10-12 23:43:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 179339264. Throughput: 0: 1637.1, 1: 1650.6. Samples: 44848440. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:43:26,443][43579] Avg episode reward: [(0, '276.440'), (1, '279.600')] [2023-10-12 23:43:26,493][44959] Updated weights for policy 1, policy_version 87790 (0.0010) [2023-10-12 23:43:26,852][44959] Updated weights for policy 1, policy_version 87800 (0.0009) [2023-10-12 23:43:26,986][44958] Updated weights for policy 0, policy_version 87370 (0.0007) [2023-10-12 23:43:27,358][44958] Updated weights for policy 0, policy_version 87380 (0.0007) [2023-10-12 23:43:27,731][44958] Updated weights for policy 0, policy_version 87390 (0.0007) [2023-10-12 23:43:31,066][44959] Updated weights for policy 1, policy_version 87810 (0.0009) [2023-10-12 23:43:31,437][44959] Updated weights for policy 1, policy_version 87820 (0.0008) [2023-10-12 23:43:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179404800. Throughput: 0: 1642.8, 1: 1646.3. Samples: 44868598. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-12 23:43:31,443][43579] Avg episode reward: [(0, '274.300'), (1, '286.540')] [2023-10-12 23:43:31,807][44959] Updated weights for policy 1, policy_version 87830 (0.0007) [2023-10-12 23:43:31,878][44958] Updated weights for policy 0, policy_version 87400 (0.0008) [2023-10-12 23:43:32,170][44959] Updated weights for policy 1, policy_version 87840 (0.0009) [2023-10-12 23:43:32,242][44958] Updated weights for policy 0, policy_version 87410 (0.0008) [2023-10-12 23:43:32,613][44958] Updated weights for policy 0, policy_version 87420 (0.0007) [2023-10-12 23:43:36,232][44959] Updated weights for policy 1, policy_version 87850 (0.0008) [2023-10-12 23:43:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179470336. Throughput: 0: 1647.6, 1: 1648.7. Samples: 44877578. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:43:36,443][43579] Avg episode reward: [(0, '266.310'), (1, '286.480')] [2023-10-12 23:43:36,600][44959] Updated weights for policy 1, policy_version 87860 (0.0009) [2023-10-12 23:43:36,864][44958] Updated weights for policy 0, policy_version 87430 (0.0008) [2023-10-12 23:43:36,960][44959] Updated weights for policy 1, policy_version 87870 (0.0008) [2023-10-12 23:43:37,236][44958] Updated weights for policy 0, policy_version 87440 (0.0008) [2023-10-12 23:43:37,617][44958] Updated weights for policy 0, policy_version 87450 (0.0008) [2023-10-12 23:43:41,288][44959] Updated weights for policy 1, policy_version 87880 (0.0008) [2023-10-12 23:43:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179535872. Throughput: 0: 1644.8, 1: 1652.0. Samples: 44897662. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:43:41,443][43579] Avg episode reward: [(0, '273.160'), (1, '286.340')] [2023-10-12 23:43:41,656][44959] Updated weights for policy 1, policy_version 87890 (0.0008) [2023-10-12 23:43:41,838][44958] Updated weights for policy 0, policy_version 87460 (0.0009) [2023-10-12 23:43:42,021][44959] Updated weights for policy 1, policy_version 87900 (0.0009) [2023-10-12 23:43:42,209][44958] Updated weights for policy 0, policy_version 87470 (0.0008) [2023-10-12 23:43:42,586][44958] Updated weights for policy 0, policy_version 87480 (0.0011) [2023-10-12 23:43:46,184][44959] Updated weights for policy 1, policy_version 87910 (0.0009) [2023-10-12 23:43:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179601408. Throughput: 0: 1644.5, 1: 1648.5. Samples: 44917880. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:43:46,443][43579] Avg episode reward: [(0, '273.020'), (1, '286.410')] [2023-10-12 23:43:46,453][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000087488_89587712.pth... [2023-10-12 23:43:46,488][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000085952_88014848.pth [2023-10-12 23:43:46,560][44959] Updated weights for policy 1, policy_version 87920 (0.0008) [2023-10-12 23:43:46,690][44958] Updated weights for policy 0, policy_version 87490 (0.0009) [2023-10-12 23:43:46,928][44959] Updated weights for policy 1, policy_version 87930 (0.0008) [2023-10-12 23:43:47,055][44958] Updated weights for policy 0, policy_version 87500 (0.0007) [2023-10-12 23:43:47,145][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000087936_90046464.pth... [2023-10-12 23:43:47,179][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000086400_88473600.pth [2023-10-12 23:43:47,416][44958] Updated weights for policy 0, policy_version 87510 (0.0009) [2023-10-12 23:43:47,784][44958] Updated weights for policy 0, policy_version 87520 (0.0011) [2023-10-12 23:43:51,033][44959] Updated weights for policy 1, policy_version 87940 (0.0008) [2023-10-12 23:43:51,402][44959] Updated weights for policy 1, policy_version 87950 (0.0009) [2023-10-12 23:43:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179666944. Throughput: 0: 1644.3, 1: 1652.1. Samples: 44926902. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:43:51,443][43579] Avg episode reward: [(0, '271.190'), (1, '282.030')] [2023-10-12 23:43:51,773][44959] Updated weights for policy 1, policy_version 87960 (0.0008) [2023-10-12 23:43:52,004][44958] Updated weights for policy 0, policy_version 87530 (0.0008) [2023-10-12 23:43:52,373][44958] Updated weights for policy 0, policy_version 87540 (0.0008) [2023-10-12 23:43:52,749][44958] Updated weights for policy 0, policy_version 87550 (0.0009) [2023-10-12 23:43:55,997][44959] Updated weights for policy 1, policy_version 87970 (0.0008) [2023-10-12 23:43:56,406][44959] Updated weights for policy 1, policy_version 87980 (0.0009) [2023-10-12 23:43:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179732480. Throughput: 0: 1636.6, 1: 1651.4. Samples: 44947076. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:43:56,443][43579] Avg episode reward: [(0, '271.950'), (1, '283.390')] [2023-10-12 23:43:56,771][44959] Updated weights for policy 1, policy_version 87990 (0.0008) [2023-10-12 23:43:57,009][44958] Updated weights for policy 0, policy_version 87560 (0.0008) [2023-10-12 23:43:57,136][44959] Updated weights for policy 1, policy_version 88000 (0.0009) [2023-10-12 23:43:57,390][44958] Updated weights for policy 0, policy_version 87570 (0.0007) [2023-10-12 23:43:57,752][44958] Updated weights for policy 0, policy_version 87580 (0.0009) [2023-10-12 23:44:01,093][44959] Updated weights for policy 1, policy_version 88010 (0.0008) [2023-10-12 23:44:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179798016. Throughput: 0: 1630.0, 1: 1647.4. Samples: 44966734. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:44:01,443][43579] Avg episode reward: [(0, '280.360'), (1, '281.020')] [2023-10-12 23:44:01,452][44959] Updated weights for policy 1, policy_version 88020 (0.0011) [2023-10-12 23:44:01,825][44959] Updated weights for policy 1, policy_version 88030 (0.0009) [2023-10-12 23:44:01,916][44958] Updated weights for policy 0, policy_version 87590 (0.0008) [2023-10-12 23:44:02,276][44958] Updated weights for policy 0, policy_version 87600 (0.0008) [2023-10-12 23:44:02,644][44958] Updated weights for policy 0, policy_version 87610 (0.0007) [2023-10-12 23:44:06,203][44959] Updated weights for policy 1, policy_version 88040 (0.0009) [2023-10-12 23:44:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179863552. Throughput: 0: 1633.3, 1: 1656.8. Samples: 44976216. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:44:06,443][43579] Avg episode reward: [(0, '279.510'), (1, '282.760')] [2023-10-12 23:44:06,565][44959] Updated weights for policy 1, policy_version 88050 (0.0008) [2023-10-12 23:44:06,907][44958] Updated weights for policy 0, policy_version 87620 (0.0008) [2023-10-12 23:44:06,933][44959] Updated weights for policy 1, policy_version 88060 (0.0010) [2023-10-12 23:44:07,280][44958] Updated weights for policy 0, policy_version 87630 (0.0008) [2023-10-12 23:44:07,656][44958] Updated weights for policy 0, policy_version 87640 (0.0007) [2023-10-12 23:44:11,114][44959] Updated weights for policy 1, policy_version 88070 (0.0009) [2023-10-12 23:44:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179929088. Throughput: 0: 1636.7, 1: 1649.3. Samples: 44996310. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:44:11,444][43579] Avg episode reward: [(0, '278.650'), (1, '284.440')] [2023-10-12 23:44:11,478][44959] Updated weights for policy 1, policy_version 88080 (0.0008) [2023-10-12 23:44:11,783][44958] Updated weights for policy 0, policy_version 87650 (0.0011) [2023-10-12 23:44:11,841][44959] Updated weights for policy 1, policy_version 88090 (0.0008) [2023-10-12 23:44:12,153][44958] Updated weights for policy 0, policy_version 87660 (0.0008) [2023-10-12 23:44:12,525][44958] Updated weights for policy 0, policy_version 87670 (0.0007) [2023-10-12 23:44:12,892][44958] Updated weights for policy 0, policy_version 87680 (0.0008) [2023-10-12 23:44:15,907][44959] Updated weights for policy 1, policy_version 88100 (0.0008) [2023-10-12 23:44:16,281][44959] Updated weights for policy 1, policy_version 88110 (0.0011) [2023-10-12 23:44:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 179994624. Throughput: 0: 1636.7, 1: 1648.8. Samples: 45016444. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:44:16,443][43579] Avg episode reward: [(0, '277.880'), (1, '286.660')] [2023-10-12 23:44:16,644][44959] Updated weights for policy 1, policy_version 88120 (0.0008) [2023-10-12 23:44:17,132][44958] Updated weights for policy 0, policy_version 87690 (0.0007) [2023-10-12 23:44:17,496][44958] Updated weights for policy 0, policy_version 87700 (0.0008) [2023-10-12 23:44:17,873][44958] Updated weights for policy 0, policy_version 87710 (0.0007) [2023-10-12 23:44:20,837][44959] Updated weights for policy 1, policy_version 88130 (0.0007) [2023-10-12 23:44:21,205][44959] Updated weights for policy 1, policy_version 88140 (0.0008) [2023-10-12 23:44:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180060160. Throughput: 0: 1635.6, 1: 1654.2. Samples: 45025620. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:44:21,444][43579] Avg episode reward: [(0, '275.650'), (1, '289.940')] [2023-10-12 23:44:21,576][44959] Updated weights for policy 1, policy_version 88150 (0.0007) [2023-10-12 23:44:21,942][44959] Updated weights for policy 1, policy_version 88160 (0.0009) [2023-10-12 23:44:22,086][44958] Updated weights for policy 0, policy_version 87720 (0.0008) [2023-10-12 23:44:22,447][44958] Updated weights for policy 0, policy_version 87730 (0.0009) [2023-10-12 23:44:22,819][44958] Updated weights for policy 0, policy_version 87740 (0.0011) [2023-10-12 23:44:26,242][44959] Updated weights for policy 1, policy_version 88170 (0.0008) [2023-10-12 23:44:26,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180125696. Throughput: 0: 1639.8, 1: 1654.2. Samples: 45045892. Policy #0 lag: (min: 17.0, avg: 22.6, max: 49.0) [2023-10-12 23:44:26,444][43579] Avg episode reward: [(0, '276.940'), (1, '289.910')] [2023-10-12 23:44:26,607][44959] Updated weights for policy 1, policy_version 88180 (0.0009) [2023-10-12 23:44:26,970][44959] Updated weights for policy 1, policy_version 88190 (0.0009) [2023-10-12 23:44:27,103][44958] Updated weights for policy 0, policy_version 87750 (0.0008) [2023-10-12 23:44:27,483][44958] Updated weights for policy 0, policy_version 87760 (0.0009) [2023-10-12 23:44:27,862][44958] Updated weights for policy 0, policy_version 87770 (0.0007) [2023-10-12 23:44:30,998][44959] Updated weights for policy 1, policy_version 88200 (0.0008) [2023-10-12 23:44:31,364][44959] Updated weights for policy 1, policy_version 88210 (0.0009) [2023-10-12 23:44:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180191232. Throughput: 0: 1640.9, 1: 1647.9. Samples: 45065872. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:44:31,443][43579] Avg episode reward: [(0, '277.250'), (1, '282.350')] [2023-10-12 23:44:31,729][44959] Updated weights for policy 1, policy_version 88220 (0.0008) [2023-10-12 23:44:31,982][44958] Updated weights for policy 0, policy_version 87780 (0.0009) [2023-10-12 23:44:32,361][44958] Updated weights for policy 0, policy_version 87790 (0.0008) [2023-10-12 23:44:32,729][44958] Updated weights for policy 0, policy_version 87800 (0.0009) [2023-10-12 23:44:35,953][44959] Updated weights for policy 1, policy_version 88230 (0.0008) [2023-10-12 23:44:36,321][44959] Updated weights for policy 1, policy_version 88240 (0.0007) [2023-10-12 23:44:36,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180256768. Throughput: 0: 1642.0, 1: 1649.5. Samples: 45075020. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:44:36,443][43579] Avg episode reward: [(0, '277.390'), (1, '279.900')] [2023-10-12 23:44:36,692][44959] Updated weights for policy 1, policy_version 88250 (0.0010) [2023-10-12 23:44:36,842][44958] Updated weights for policy 0, policy_version 87810 (0.0009) [2023-10-12 23:44:37,209][44958] Updated weights for policy 0, policy_version 87820 (0.0007) [2023-10-12 23:44:37,573][44958] Updated weights for policy 0, policy_version 87830 (0.0010) [2023-10-12 23:44:37,945][44958] Updated weights for policy 0, policy_version 87840 (0.0011) [2023-10-12 23:44:40,758][44959] Updated weights for policy 1, policy_version 88260 (0.0007) [2023-10-12 23:44:41,158][44959] Updated weights for policy 1, policy_version 88270 (0.0007) [2023-10-12 23:44:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180322304. Throughput: 0: 1645.3, 1: 1652.6. Samples: 45095484. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:44:41,444][43579] Avg episode reward: [(0, '278.790'), (1, '279.550')] [2023-10-12 23:44:41,523][44959] Updated weights for policy 1, policy_version 88280 (0.0008) [2023-10-12 23:44:41,985][44958] Updated weights for policy 0, policy_version 87850 (0.0008) [2023-10-12 23:44:42,353][44958] Updated weights for policy 0, policy_version 87860 (0.0007) [2023-10-12 23:44:42,725][44958] Updated weights for policy 0, policy_version 87870 (0.0008) [2023-10-12 23:44:45,566][44959] Updated weights for policy 1, policy_version 88290 (0.0009) [2023-10-12 23:44:45,941][44959] Updated weights for policy 1, policy_version 88300 (0.0007) [2023-10-12 23:44:46,306][44959] Updated weights for policy 1, policy_version 88310 (0.0007) [2023-10-12 23:44:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180387840. Throughput: 0: 1652.2, 1: 1646.8. Samples: 45115188. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:44:46,443][43579] Avg episode reward: [(0, '273.900'), (1, '279.580')] [2023-10-12 23:44:46,674][44959] Updated weights for policy 1, policy_version 88320 (0.0009) [2023-10-12 23:44:46,819][44958] Updated weights for policy 0, policy_version 87880 (0.0008) [2023-10-12 23:44:47,191][44958] Updated weights for policy 0, policy_version 87890 (0.0011) [2023-10-12 23:44:47,564][44958] Updated weights for policy 0, policy_version 87900 (0.0010) [2023-10-12 23:44:50,751][44959] Updated weights for policy 1, policy_version 88330 (0.0009) [2023-10-12 23:44:51,119][44959] Updated weights for policy 1, policy_version 88340 (0.0008) [2023-10-12 23:44:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180453376. Throughput: 0: 1648.1, 1: 1646.5. Samples: 45124472. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:44:51,443][43579] Avg episode reward: [(0, '270.890'), (1, '275.600')] [2023-10-12 23:44:51,484][44959] Updated weights for policy 1, policy_version 88350 (0.0009) [2023-10-12 23:44:51,577][44958] Updated weights for policy 0, policy_version 87910 (0.0007) [2023-10-12 23:44:51,942][44958] Updated weights for policy 0, policy_version 87920 (0.0008) [2023-10-12 23:44:52,327][44958] Updated weights for policy 0, policy_version 87930 (0.0009) [2023-10-12 23:44:55,768][44959] Updated weights for policy 1, policy_version 88360 (0.0008) [2023-10-12 23:44:56,132][44959] Updated weights for policy 1, policy_version 88370 (0.0010) [2023-10-12 23:44:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180518912. Throughput: 0: 1648.9, 1: 1648.4. Samples: 45144688. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:44:56,443][43579] Avg episode reward: [(0, '268.720'), (1, '280.310')] [2023-10-12 23:44:56,497][44959] Updated weights for policy 1, policy_version 88380 (0.0009) [2023-10-12 23:44:56,671][44958] Updated weights for policy 0, policy_version 87940 (0.0007) [2023-10-12 23:44:57,043][44958] Updated weights for policy 0, policy_version 87950 (0.0008) [2023-10-12 23:44:57,417][44958] Updated weights for policy 0, policy_version 87960 (0.0007) [2023-10-12 23:45:00,732][44959] Updated weights for policy 1, policy_version 88390 (0.0010) [2023-10-12 23:45:01,099][44959] Updated weights for policy 1, policy_version 88400 (0.0010) [2023-10-12 23:45:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180584448. Throughput: 0: 1647.2, 1: 1636.5. Samples: 45164212. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:45:01,443][43579] Avg episode reward: [(0, '267.850'), (1, '283.090')] [2023-10-12 23:45:01,474][44959] Updated weights for policy 1, policy_version 88410 (0.0010) [2023-10-12 23:45:01,630][44958] Updated weights for policy 0, policy_version 87970 (0.0009) [2023-10-12 23:45:02,016][44958] Updated weights for policy 0, policy_version 87980 (0.0009) [2023-10-12 23:45:02,393][44958] Updated weights for policy 0, policy_version 87990 (0.0007) [2023-10-12 23:45:02,763][44958] Updated weights for policy 0, policy_version 88000 (0.0008) [2023-10-12 23:45:05,663][44959] Updated weights for policy 1, policy_version 88420 (0.0009) [2023-10-12 23:45:06,038][44959] Updated weights for policy 1, policy_version 88430 (0.0009) [2023-10-12 23:45:06,405][44959] Updated weights for policy 1, policy_version 88440 (0.0010) [2023-10-12 23:45:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180649984. Throughput: 0: 1648.4, 1: 1639.5. Samples: 45173574. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:45:06,443][43579] Avg episode reward: [(0, '265.470'), (1, '283.650')] [2023-10-12 23:45:06,749][44958] Updated weights for policy 0, policy_version 88010 (0.0009) [2023-10-12 23:45:07,118][44958] Updated weights for policy 0, policy_version 88020 (0.0011) [2023-10-12 23:45:07,488][44958] Updated weights for policy 0, policy_version 88030 (0.0010) [2023-10-12 23:45:10,623][44959] Updated weights for policy 1, policy_version 88450 (0.0008) [2023-10-12 23:45:10,996][44959] Updated weights for policy 1, policy_version 88460 (0.0008) [2023-10-12 23:45:11,357][44959] Updated weights for policy 1, policy_version 88470 (0.0007) [2023-10-12 23:45:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180715520. Throughput: 0: 1647.7, 1: 1637.1. Samples: 45193708. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:45:11,443][43579] Avg episode reward: [(0, '265.940'), (1, '284.580')] [2023-10-12 23:45:11,726][44959] Updated weights for policy 1, policy_version 88480 (0.0007) [2023-10-12 23:45:11,726][44958] Updated weights for policy 0, policy_version 88040 (0.0009) [2023-10-12 23:45:12,102][44958] Updated weights for policy 0, policy_version 88050 (0.0008) [2023-10-12 23:45:12,471][44958] Updated weights for policy 0, policy_version 88060 (0.0008) [2023-10-12 23:45:15,989][44959] Updated weights for policy 1, policy_version 88490 (0.0007) [2023-10-12 23:45:16,361][44959] Updated weights for policy 1, policy_version 88500 (0.0007) [2023-10-12 23:45:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180781056. Throughput: 0: 1649.6, 1: 1633.5. Samples: 45213610. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:45:16,443][43579] Avg episode reward: [(0, '270.300'), (1, '285.600')] [2023-10-12 23:45:16,646][44958] Updated weights for policy 0, policy_version 88070 (0.0008) [2023-10-12 23:45:16,726][44959] Updated weights for policy 1, policy_version 88510 (0.0008) [2023-10-12 23:45:17,017][44958] Updated weights for policy 0, policy_version 88080 (0.0007) [2023-10-12 23:45:17,399][44958] Updated weights for policy 0, policy_version 88090 (0.0007) [2023-10-12 23:45:20,913][44959] Updated weights for policy 1, policy_version 88520 (0.0007) [2023-10-12 23:45:21,282][44959] Updated weights for policy 1, policy_version 88530 (0.0008) [2023-10-12 23:45:21,351][44958] Updated weights for policy 0, policy_version 88100 (0.0008) [2023-10-12 23:45:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180846592. Throughput: 0: 1650.3, 1: 1638.9. Samples: 45223034. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-12 23:45:21,443][43579] Avg episode reward: [(0, '265.840'), (1, '283.750')] [2023-10-12 23:45:21,654][44959] Updated weights for policy 1, policy_version 88540 (0.0007) [2023-10-12 23:45:21,720][44958] Updated weights for policy 0, policy_version 88110 (0.0009) [2023-10-12 23:45:22,095][44958] Updated weights for policy 0, policy_version 88120 (0.0009) [2023-10-12 23:45:25,905][44959] Updated weights for policy 1, policy_version 88550 (0.0009) [2023-10-12 23:45:26,282][44959] Updated weights for policy 1, policy_version 88560 (0.0009) [2023-10-12 23:45:26,340][44958] Updated weights for policy 0, policy_version 88130 (0.0010) [2023-10-12 23:45:26,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180912128. Throughput: 0: 1650.4, 1: 1629.2. Samples: 45243066. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:45:26,443][43579] Avg episode reward: [(0, '272.560'), (1, '284.740')] [2023-10-12 23:45:26,645][44959] Updated weights for policy 1, policy_version 88570 (0.0008) [2023-10-12 23:45:26,742][44958] Updated weights for policy 0, policy_version 88140 (0.0009) [2023-10-12 23:45:27,120][44958] Updated weights for policy 0, policy_version 88150 (0.0008) [2023-10-12 23:45:27,494][44958] Updated weights for policy 0, policy_version 88160 (0.0007) [2023-10-12 23:45:30,783][44959] Updated weights for policy 1, policy_version 88580 (0.0008) [2023-10-12 23:45:31,158][44959] Updated weights for policy 1, policy_version 88590 (0.0008) [2023-10-12 23:45:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 180977664. Throughput: 0: 1647.7, 1: 1627.2. Samples: 45262558. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:45:31,443][43579] Avg episode reward: [(0, '273.560'), (1, '285.880')] [2023-10-12 23:45:31,528][44959] Updated weights for policy 1, policy_version 88600 (0.0009) [2023-10-12 23:45:31,654][44958] Updated weights for policy 0, policy_version 88170 (0.0008) [2023-10-12 23:45:32,035][44958] Updated weights for policy 0, policy_version 88180 (0.0009) [2023-10-12 23:45:32,411][44958] Updated weights for policy 0, policy_version 88190 (0.0008) [2023-10-12 23:45:35,884][44959] Updated weights for policy 1, policy_version 88610 (0.0009) [2023-10-12 23:45:36,262][44959] Updated weights for policy 1, policy_version 88620 (0.0009) [2023-10-12 23:45:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181043200. Throughput: 0: 1649.2, 1: 1625.9. Samples: 45271852. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:45:36,444][43579] Avg episode reward: [(0, '275.260'), (1, '285.420')] [2023-10-12 23:45:36,475][44958] Updated weights for policy 0, policy_version 88200 (0.0009) [2023-10-12 23:45:36,623][44959] Updated weights for policy 1, policy_version 88630 (0.0009) [2023-10-12 23:45:36,850][44958] Updated weights for policy 0, policy_version 88210 (0.0009) [2023-10-12 23:45:36,989][44959] Updated weights for policy 1, policy_version 88640 (0.0007) [2023-10-12 23:45:37,222][44958] Updated weights for policy 0, policy_version 88220 (0.0008) [2023-10-12 23:45:41,126][44959] Updated weights for policy 1, policy_version 88650 (0.0007) [2023-10-12 23:45:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181108736. Throughput: 0: 1640.4, 1: 1626.2. Samples: 45291686. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:45:41,443][43579] Avg episode reward: [(0, '278.350'), (1, '284.000')] [2023-10-12 23:45:41,491][44959] Updated weights for policy 1, policy_version 88660 (0.0009) [2023-10-12 23:45:41,572][44958] Updated weights for policy 0, policy_version 88230 (0.0008) [2023-10-12 23:45:41,858][44959] Updated weights for policy 1, policy_version 88670 (0.0007) [2023-10-12 23:45:41,942][44958] Updated weights for policy 0, policy_version 88240 (0.0007) [2023-10-12 23:45:42,312][44958] Updated weights for policy 0, policy_version 88250 (0.0009) [2023-10-12 23:45:45,967][44959] Updated weights for policy 1, policy_version 88680 (0.0009) [2023-10-12 23:45:46,347][44959] Updated weights for policy 1, policy_version 88690 (0.0007) [2023-10-12 23:45:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181174272. Throughput: 0: 1638.7, 1: 1639.3. Samples: 45311722. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:45:46,443][43579] Avg episode reward: [(0, '276.080'), (1, '284.070')] [2023-10-12 23:45:46,624][44958] Updated weights for policy 0, policy_version 88260 (0.0009) [2023-10-12 23:45:46,712][44959] Updated weights for policy 1, policy_version 88700 (0.0007) [2023-10-12 23:45:46,859][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000088704_90832896.pth... [2023-10-12 23:45:46,894][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000087136_89227264.pth [2023-10-12 23:45:46,988][44958] Updated weights for policy 0, policy_version 88270 (0.0009) [2023-10-12 23:45:47,366][44958] Updated weights for policy 0, policy_version 88280 (0.0011) [2023-10-12 23:45:47,657][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000088288_90406912.pth... [2023-10-12 23:45:47,696][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000086752_88834048.pth [2023-10-12 23:45:51,037][44959] Updated weights for policy 1, policy_version 88710 (0.0009) [2023-10-12 23:45:51,402][44959] Updated weights for policy 1, policy_version 88720 (0.0010) [2023-10-12 23:45:51,431][44958] Updated weights for policy 0, policy_version 88290 (0.0007) [2023-10-12 23:45:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181239808. Throughput: 0: 1640.3, 1: 1635.0. Samples: 45320962. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:45:51,443][43579] Avg episode reward: [(0, '283.020'), (1, '285.810')] [2023-10-12 23:45:51,771][44959] Updated weights for policy 1, policy_version 88730 (0.0007) [2023-10-12 23:45:51,797][44958] Updated weights for policy 0, policy_version 88300 (0.0007) [2023-10-12 23:45:52,163][44958] Updated weights for policy 0, policy_version 88310 (0.0009) [2023-10-12 23:45:52,539][44958] Updated weights for policy 0, policy_version 88320 (0.0010) [2023-10-12 23:45:55,792][44959] Updated weights for policy 1, policy_version 88740 (0.0007) [2023-10-12 23:45:56,153][44959] Updated weights for policy 1, policy_version 88750 (0.0007) [2023-10-12 23:45:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181305344. Throughput: 0: 1640.6, 1: 1638.3. Samples: 45341260. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:45:56,443][43579] Avg episode reward: [(0, '281.930'), (1, '284.960')] [2023-10-12 23:45:56,523][44959] Updated weights for policy 1, policy_version 88760 (0.0007) [2023-10-12 23:45:56,783][44958] Updated weights for policy 0, policy_version 88330 (0.0007) [2023-10-12 23:45:57,147][44958] Updated weights for policy 0, policy_version 88340 (0.0007) [2023-10-12 23:45:57,529][44958] Updated weights for policy 0, policy_version 88350 (0.0009) [2023-10-12 23:46:00,593][44959] Updated weights for policy 1, policy_version 88770 (0.0007) [2023-10-12 23:46:00,950][44959] Updated weights for policy 1, policy_version 88780 (0.0008) [2023-10-12 23:46:01,324][44959] Updated weights for policy 1, policy_version 88790 (0.0010) [2023-10-12 23:46:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181370880. Throughput: 0: 1635.5, 1: 1638.8. Samples: 45360956. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:46:01,443][43579] Avg episode reward: [(0, '282.710'), (1, '277.690')] [2023-10-12 23:46:01,601][44958] Updated weights for policy 0, policy_version 88360 (0.0009) [2023-10-12 23:46:01,688][44959] Updated weights for policy 1, policy_version 88800 (0.0008) [2023-10-12 23:46:01,957][44958] Updated weights for policy 0, policy_version 88370 (0.0010) [2023-10-12 23:46:02,335][44958] Updated weights for policy 0, policy_version 88380 (0.0011) [2023-10-12 23:46:05,982][44959] Updated weights for policy 1, policy_version 88810 (0.0009) [2023-10-12 23:46:06,352][44959] Updated weights for policy 1, policy_version 88820 (0.0007) [2023-10-12 23:46:06,352][44958] Updated weights for policy 0, policy_version 88390 (0.0008) [2023-10-12 23:46:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181436416. Throughput: 0: 1633.9, 1: 1637.4. Samples: 45370240. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:46:06,444][43579] Avg episode reward: [(0, '282.750'), (1, '280.620')] [2023-10-12 23:46:06,718][44959] Updated weights for policy 1, policy_version 88830 (0.0008) [2023-10-12 23:46:06,720][44958] Updated weights for policy 0, policy_version 88400 (0.0008) [2023-10-12 23:46:07,097][44958] Updated weights for policy 0, policy_version 88410 (0.0010) [2023-10-12 23:46:10,977][44959] Updated weights for policy 1, policy_version 88840 (0.0007) [2023-10-12 23:46:11,349][44959] Updated weights for policy 1, policy_version 88850 (0.0008) [2023-10-12 23:46:11,437][44958] Updated weights for policy 0, policy_version 88420 (0.0009) [2023-10-12 23:46:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181501952. Throughput: 0: 1636.5, 1: 1640.8. Samples: 45390546. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:46:11,443][43579] Avg episode reward: [(0, '280.180'), (1, '278.370')] [2023-10-12 23:46:11,723][44959] Updated weights for policy 1, policy_version 88860 (0.0008) [2023-10-12 23:46:11,820][44958] Updated weights for policy 0, policy_version 88430 (0.0009) [2023-10-12 23:46:12,186][44958] Updated weights for policy 0, policy_version 88440 (0.0007) [2023-10-12 23:46:15,661][44959] Updated weights for policy 1, policy_version 88870 (0.0007) [2023-10-12 23:46:16,025][44959] Updated weights for policy 1, policy_version 88880 (0.0008) [2023-10-12 23:46:16,376][44958] Updated weights for policy 0, policy_version 88450 (0.0008) [2023-10-12 23:46:16,387][44959] Updated weights for policy 1, policy_version 88890 (0.0009) [2023-10-12 23:46:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181567488. Throughput: 0: 1641.3, 1: 1645.9. Samples: 45410486. Policy #0 lag: (min: 33.0, avg: 53.9, max: 56.0) [2023-10-12 23:46:16,444][43579] Avg episode reward: [(0, '283.740'), (1, '277.680')] [2023-10-12 23:46:16,749][44958] Updated weights for policy 0, policy_version 88460 (0.0009) [2023-10-12 23:46:17,119][44958] Updated weights for policy 0, policy_version 88470 (0.0009) [2023-10-12 23:46:17,505][44958] Updated weights for policy 0, policy_version 88480 (0.0010) [2023-10-12 23:46:20,598][44959] Updated weights for policy 1, policy_version 88900 (0.0008) [2023-10-12 23:46:20,971][44959] Updated weights for policy 1, policy_version 88910 (0.0009) [2023-10-12 23:46:21,352][44959] Updated weights for policy 1, policy_version 88920 (0.0007) [2023-10-12 23:46:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181633024. Throughput: 0: 1637.0, 1: 1652.9. Samples: 45419896. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:46:21,443][43579] Avg episode reward: [(0, '282.200'), (1, '280.140')] [2023-10-12 23:46:21,677][44958] Updated weights for policy 0, policy_version 88490 (0.0008) [2023-10-12 23:46:22,044][44958] Updated weights for policy 0, policy_version 88500 (0.0007) [2023-10-12 23:46:22,418][44958] Updated weights for policy 0, policy_version 88510 (0.0008) [2023-10-12 23:46:25,495][44959] Updated weights for policy 1, policy_version 88930 (0.0008) [2023-10-12 23:46:25,875][44959] Updated weights for policy 1, policy_version 88940 (0.0008) [2023-10-12 23:46:26,244][44959] Updated weights for policy 1, policy_version 88950 (0.0008) [2023-10-12 23:46:26,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181698560. Throughput: 0: 1640.4, 1: 1654.0. Samples: 45439934. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:46:26,443][43579] Avg episode reward: [(0, '278.860'), (1, '280.960')] [2023-10-12 23:46:26,604][44959] Updated weights for policy 1, policy_version 88960 (0.0009) [2023-10-12 23:46:26,638][44958] Updated weights for policy 0, policy_version 88520 (0.0008) [2023-10-12 23:46:27,008][44958] Updated weights for policy 0, policy_version 88530 (0.0009) [2023-10-12 23:46:27,377][44958] Updated weights for policy 0, policy_version 88540 (0.0007) [2023-10-12 23:46:30,644][44959] Updated weights for policy 1, policy_version 88970 (0.0008) [2023-10-12 23:46:31,017][44959] Updated weights for policy 1, policy_version 88980 (0.0008) [2023-10-12 23:46:31,381][44959] Updated weights for policy 1, policy_version 88990 (0.0008) [2023-10-12 23:46:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 181764096. Throughput: 0: 1642.5, 1: 1643.3. Samples: 45459586. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:46:31,443][43579] Avg episode reward: [(0, '273.350'), (1, '288.010')] [2023-10-12 23:46:31,708][44958] Updated weights for policy 0, policy_version 88550 (0.0009) [2023-10-12 23:46:32,078][44958] Updated weights for policy 0, policy_version 88560 (0.0008) [2023-10-12 23:46:32,453][44958] Updated weights for policy 0, policy_version 88570 (0.0009) [2023-10-12 23:46:35,559][44959] Updated weights for policy 1, policy_version 89000 (0.0008) [2023-10-12 23:46:35,922][44959] Updated weights for policy 1, policy_version 89010 (0.0010) [2023-10-12 23:46:36,295][44959] Updated weights for policy 1, policy_version 89020 (0.0009) [2023-10-12 23:46:36,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 181862400. Throughput: 0: 1638.5, 1: 1656.6. Samples: 45469242. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:46:36,443][43579] Avg episode reward: [(0, '272.850'), (1, '288.860')] [2023-10-12 23:46:36,566][44958] Updated weights for policy 0, policy_version 88580 (0.0008) [2023-10-12 23:46:36,930][44958] Updated weights for policy 0, policy_version 88590 (0.0009) [2023-10-12 23:46:37,298][44958] Updated weights for policy 0, policy_version 88600 (0.0008) [2023-10-12 23:46:40,447][44959] Updated weights for policy 1, policy_version 89030 (0.0011) [2023-10-12 23:46:40,813][44959] Updated weights for policy 1, policy_version 89040 (0.0009) [2023-10-12 23:46:41,186][44959] Updated weights for policy 1, policy_version 89050 (0.0007) [2023-10-12 23:46:41,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 181927936. Throughput: 0: 1634.5, 1: 1654.0. Samples: 45489246. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:46:41,443][43579] Avg episode reward: [(0, '274.170'), (1, '290.890')] [2023-10-12 23:46:41,631][44958] Updated weights for policy 0, policy_version 88610 (0.0007) [2023-10-12 23:46:41,997][44958] Updated weights for policy 0, policy_version 88620 (0.0011) [2023-10-12 23:46:42,372][44958] Updated weights for policy 0, policy_version 88630 (0.0009) [2023-10-12 23:46:42,746][44958] Updated weights for policy 0, policy_version 88640 (0.0009) [2023-10-12 23:46:45,245][44959] Updated weights for policy 1, policy_version 89060 (0.0009) [2023-10-12 23:46:45,610][44959] Updated weights for policy 1, policy_version 89070 (0.0009) [2023-10-12 23:46:45,982][44959] Updated weights for policy 1, policy_version 89080 (0.0009) [2023-10-12 23:46:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 181993472. Throughput: 0: 1642.4, 1: 1646.1. Samples: 45508942. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:46:46,443][43579] Avg episode reward: [(0, '269.510'), (1, '292.590')] [2023-10-12 23:46:46,788][44958] Updated weights for policy 0, policy_version 88650 (0.0007) [2023-10-12 23:46:47,161][44958] Updated weights for policy 0, policy_version 88660 (0.0008) [2023-10-12 23:46:47,535][44958] Updated weights for policy 0, policy_version 88670 (0.0007) [2023-10-12 23:46:50,321][44959] Updated weights for policy 1, policy_version 89090 (0.0010) [2023-10-12 23:46:50,692][44959] Updated weights for policy 1, policy_version 89100 (0.0010) [2023-10-12 23:46:51,055][44959] Updated weights for policy 1, policy_version 89110 (0.0010) [2023-10-12 23:46:51,419][44959] Updated weights for policy 1, policy_version 89120 (0.0008) [2023-10-12 23:46:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 182059008. Throughput: 0: 1640.4, 1: 1652.5. Samples: 45518422. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:46:51,444][43579] Avg episode reward: [(0, '273.180'), (1, '286.600')] [2023-10-12 23:46:51,656][44958] Updated weights for policy 0, policy_version 88680 (0.0010) [2023-10-12 23:46:52,030][44958] Updated weights for policy 0, policy_version 88690 (0.0010) [2023-10-12 23:46:52,390][44958] Updated weights for policy 0, policy_version 88700 (0.0011) [2023-10-12 23:46:55,744][44959] Updated weights for policy 1, policy_version 89130 (0.0008) [2023-10-12 23:46:56,123][44959] Updated weights for policy 1, policy_version 89140 (0.0007) [2023-10-12 23:46:56,442][43579] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 182091776. Throughput: 0: 1632.5, 1: 1656.4. Samples: 45538548. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:46:56,443][43579] Avg episode reward: [(0, '275.870'), (1, '283.780')] [2023-10-12 23:46:56,488][44959] Updated weights for policy 1, policy_version 89150 (0.0008) [2023-10-12 23:46:56,766][44958] Updated weights for policy 0, policy_version 88710 (0.0008) [2023-10-12 23:46:57,144][44958] Updated weights for policy 0, policy_version 88720 (0.0007) [2023-10-12 23:46:57,520][44958] Updated weights for policy 0, policy_version 88730 (0.0009) [2023-10-12 23:47:00,564][44959] Updated weights for policy 1, policy_version 89160 (0.0008) [2023-10-12 23:47:00,927][44959] Updated weights for policy 1, policy_version 89170 (0.0009) [2023-10-12 23:47:01,292][44959] Updated weights for policy 1, policy_version 89180 (0.0008) [2023-10-12 23:47:01,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 182190080. Throughput: 0: 1630.6, 1: 1649.3. Samples: 45558080. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:47:01,443][43579] Avg episode reward: [(0, '278.020'), (1, '278.500')] [2023-10-12 23:47:01,586][44958] Updated weights for policy 0, policy_version 88740 (0.0010) [2023-10-12 23:47:01,960][44958] Updated weights for policy 0, policy_version 88750 (0.0009) [2023-10-12 23:47:02,329][44958] Updated weights for policy 0, policy_version 88760 (0.0007) [2023-10-12 23:47:05,334][44959] Updated weights for policy 1, policy_version 89190 (0.0010) [2023-10-12 23:47:05,709][44959] Updated weights for policy 1, policy_version 89200 (0.0007) [2023-10-12 23:47:06,080][44959] Updated weights for policy 1, policy_version 89210 (0.0008) [2023-10-12 23:47:06,392][44958] Updated weights for policy 0, policy_version 88770 (0.0009) [2023-10-12 23:47:06,443][43579] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 182255616. Throughput: 0: 1634.3, 1: 1654.4. Samples: 45567890. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:47:06,444][43579] Avg episode reward: [(0, '278.360'), (1, '279.970')] [2023-10-12 23:47:06,772][44958] Updated weights for policy 0, policy_version 88780 (0.0009) [2023-10-12 23:47:07,143][44958] Updated weights for policy 0, policy_version 88790 (0.0011) [2023-10-12 23:47:07,512][44958] Updated weights for policy 0, policy_version 88800 (0.0011) [2023-10-12 23:47:10,079][44959] Updated weights for policy 1, policy_version 89220 (0.0010) [2023-10-12 23:47:10,445][44959] Updated weights for policy 1, policy_version 89230 (0.0007) [2023-10-12 23:47:10,817][44959] Updated weights for policy 1, policy_version 89240 (0.0010) [2023-10-12 23:47:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 182321152. Throughput: 0: 1637.0, 1: 1654.1. Samples: 45588036. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-12 23:47:11,443][43579] Avg episode reward: [(0, '278.260'), (1, '279.360')] [2023-10-12 23:47:11,940][44958] Updated weights for policy 0, policy_version 88810 (0.0007) [2023-10-12 23:47:12,320][44958] Updated weights for policy 0, policy_version 88820 (0.0008) [2023-10-12 23:47:12,684][44958] Updated weights for policy 0, policy_version 88830 (0.0007) [2023-10-12 23:47:14,993][44959] Updated weights for policy 1, policy_version 89250 (0.0009) [2023-10-12 23:47:15,373][44959] Updated weights for policy 1, policy_version 89260 (0.0008) [2023-10-12 23:47:15,738][44959] Updated weights for policy 1, policy_version 89270 (0.0009) [2023-10-12 23:47:16,106][44959] Updated weights for policy 1, policy_version 89280 (0.0009) [2023-10-12 23:47:16,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 182386688. Throughput: 0: 1639.7, 1: 1640.3. Samples: 45607186. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:47:16,444][43579] Avg episode reward: [(0, '277.800'), (1, '279.150')] [2023-10-12 23:47:16,745][44958] Updated weights for policy 0, policy_version 88840 (0.0010) [2023-10-12 23:47:17,120][44958] Updated weights for policy 0, policy_version 88850 (0.0010) [2023-10-12 23:47:17,493][44958] Updated weights for policy 0, policy_version 88860 (0.0011) [2023-10-12 23:47:20,420][44959] Updated weights for policy 1, policy_version 89290 (0.0007) [2023-10-12 23:47:20,782][44959] Updated weights for policy 1, policy_version 89300 (0.0010) [2023-10-12 23:47:21,156][44959] Updated weights for policy 1, policy_version 89310 (0.0009) [2023-10-12 23:47:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 182452224. Throughput: 0: 1637.1, 1: 1645.6. Samples: 45616962. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:47:21,443][43579] Avg episode reward: [(0, '277.630'), (1, '284.460')] [2023-10-12 23:47:21,741][44958] Updated weights for policy 0, policy_version 88870 (0.0011) [2023-10-12 23:47:22,110][44958] Updated weights for policy 0, policy_version 88880 (0.0009) [2023-10-12 23:47:22,487][44958] Updated weights for policy 0, policy_version 88890 (0.0009) [2023-10-12 23:47:25,236][44959] Updated weights for policy 1, policy_version 89320 (0.0008) [2023-10-12 23:47:25,611][44959] Updated weights for policy 1, policy_version 89330 (0.0007) [2023-10-12 23:47:25,985][44959] Updated weights for policy 1, policy_version 89340 (0.0007) [2023-10-12 23:47:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 182517760. Throughput: 0: 1639.7, 1: 1644.0. Samples: 45637012. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:47:26,443][43579] Avg episode reward: [(0, '273.420'), (1, '287.810')] [2023-10-12 23:47:26,560][44958] Updated weights for policy 0, policy_version 88900 (0.0008) [2023-10-12 23:47:26,936][44958] Updated weights for policy 0, policy_version 88910 (0.0007) [2023-10-12 23:47:27,298][44958] Updated weights for policy 0, policy_version 88920 (0.0008) [2023-10-12 23:47:30,305][44959] Updated weights for policy 1, policy_version 89350 (0.0010) [2023-10-12 23:47:30,673][44959] Updated weights for policy 1, policy_version 89360 (0.0007) [2023-10-12 23:47:31,037][44959] Updated weights for policy 1, policy_version 89370 (0.0007) [2023-10-12 23:47:31,261][44958] Updated weights for policy 0, policy_version 88930 (0.0008) [2023-10-12 23:47:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 182583296. Throughput: 0: 1636.7, 1: 1640.4. Samples: 45656408. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:47:31,443][43579] Avg episode reward: [(0, '275.290'), (1, '292.690')] [2023-10-12 23:47:31,631][44958] Updated weights for policy 0, policy_version 88940 (0.0009) [2023-10-12 23:47:32,006][44958] Updated weights for policy 0, policy_version 88950 (0.0008) [2023-10-12 23:47:32,377][44958] Updated weights for policy 0, policy_version 88960 (0.0008) [2023-10-12 23:47:35,201][44959] Updated weights for policy 1, policy_version 89380 (0.0010) [2023-10-12 23:47:35,564][44959] Updated weights for policy 1, policy_version 89390 (0.0009) [2023-10-12 23:47:35,940][44959] Updated weights for policy 1, policy_version 89400 (0.0011) [2023-10-12 23:47:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 182648832. Throughput: 0: 1635.9, 1: 1646.4. Samples: 45666124. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:47:36,444][43579] Avg episode reward: [(0, '271.230'), (1, '291.460')] [2023-10-12 23:47:36,510][44958] Updated weights for policy 0, policy_version 88970 (0.0009) [2023-10-12 23:47:36,880][44958] Updated weights for policy 0, policy_version 88980 (0.0008) [2023-10-12 23:47:37,249][44958] Updated weights for policy 0, policy_version 88990 (0.0008) [2023-10-12 23:47:40,053][44959] Updated weights for policy 1, policy_version 89410 (0.0009) [2023-10-12 23:47:40,421][44959] Updated weights for policy 1, policy_version 89420 (0.0008) [2023-10-12 23:47:40,790][44959] Updated weights for policy 1, policy_version 89430 (0.0009) [2023-10-12 23:47:41,156][44959] Updated weights for policy 1, policy_version 89440 (0.0008) [2023-10-12 23:47:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 182714368. Throughput: 0: 1642.5, 1: 1646.1. Samples: 45686534. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:47:41,443][43579] Avg episode reward: [(0, '277.780'), (1, '291.480')] [2023-10-12 23:47:41,489][44958] Updated weights for policy 0, policy_version 89000 (0.0010) [2023-10-12 23:47:41,873][44958] Updated weights for policy 0, policy_version 89010 (0.0010) [2023-10-12 23:47:42,242][44958] Updated weights for policy 0, policy_version 89020 (0.0008) [2023-10-12 23:47:45,374][44959] Updated weights for policy 1, policy_version 89450 (0.0010) [2023-10-12 23:47:45,747][44959] Updated weights for policy 1, policy_version 89460 (0.0007) [2023-10-12 23:47:46,121][44959] Updated weights for policy 1, policy_version 89470 (0.0007) [2023-10-12 23:47:46,231][44958] Updated weights for policy 0, policy_version 89030 (0.0007) [2023-10-12 23:47:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 182779904. Throughput: 0: 1638.4, 1: 1637.8. Samples: 45705510. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:47:46,443][43579] Avg episode reward: [(0, '280.350'), (1, '289.210')] [2023-10-12 23:47:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000089472_91619328.pth... [2023-10-12 23:47:46,488][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000087936_90046464.pth [2023-10-12 23:47:46,593][44958] Updated weights for policy 0, policy_version 89040 (0.0010) [2023-10-12 23:47:46,968][44958] Updated weights for policy 0, policy_version 89050 (0.0007) [2023-10-12 23:47:47,185][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000089056_91193344.pth... [2023-10-12 23:47:47,226][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000087488_89587712.pth [2023-10-12 23:47:50,168][44959] Updated weights for policy 1, policy_version 89480 (0.0009) [2023-10-12 23:47:50,539][44959] Updated weights for policy 1, policy_version 89490 (0.0011) [2023-10-12 23:47:50,904][44959] Updated weights for policy 1, policy_version 89500 (0.0008) [2023-10-12 23:47:51,396][44958] Updated weights for policy 0, policy_version 89060 (0.0008) [2023-10-12 23:47:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 182845440. Throughput: 0: 1635.9, 1: 1640.1. Samples: 45715310. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:47:51,443][43579] Avg episode reward: [(0, '282.150'), (1, '289.910')] [2023-10-12 23:47:51,767][44958] Updated weights for policy 0, policy_version 89070 (0.0009) [2023-10-12 23:47:52,141][44958] Updated weights for policy 0, policy_version 89080 (0.0008) [2023-10-12 23:47:55,038][44959] Updated weights for policy 1, policy_version 89510 (0.0007) [2023-10-12 23:47:55,407][44959] Updated weights for policy 1, policy_version 89520 (0.0007) [2023-10-12 23:47:55,773][44959] Updated weights for policy 1, policy_version 89530 (0.0008) [2023-10-12 23:47:56,284][44958] Updated weights for policy 0, policy_version 89090 (0.0008) [2023-10-12 23:47:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 182910976. Throughput: 0: 1639.2, 1: 1641.1. Samples: 45735646. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:47:56,443][43579] Avg episode reward: [(0, '284.600'), (1, '289.570')] [2023-10-12 23:47:56,667][44958] Updated weights for policy 0, policy_version 89100 (0.0008) [2023-10-12 23:47:57,046][44958] Updated weights for policy 0, policy_version 89110 (0.0009) [2023-10-12 23:47:57,418][44958] Updated weights for policy 0, policy_version 89120 (0.0011) [2023-10-12 23:47:59,949][44959] Updated weights for policy 1, policy_version 89540 (0.0008) [2023-10-12 23:48:00,309][44959] Updated weights for policy 1, policy_version 89550 (0.0007) [2023-10-12 23:48:00,681][44959] Updated weights for policy 1, policy_version 89560 (0.0007) [2023-10-12 23:48:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 182976512. Throughput: 0: 1638.1, 1: 1642.5. Samples: 45754810. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:48:01,443][43579] Avg episode reward: [(0, '282.610'), (1, '288.190')] [2023-10-12 23:48:01,534][44958] Updated weights for policy 0, policy_version 89130 (0.0010) [2023-10-12 23:48:01,914][44958] Updated weights for policy 0, policy_version 89140 (0.0010) [2023-10-12 23:48:02,288][44958] Updated weights for policy 0, policy_version 89150 (0.0011) [2023-10-12 23:48:04,854][44959] Updated weights for policy 1, policy_version 89570 (0.0007) [2023-10-12 23:48:05,219][44959] Updated weights for policy 1, policy_version 89580 (0.0007) [2023-10-12 23:48:05,590][44959] Updated weights for policy 1, policy_version 89590 (0.0008) [2023-10-12 23:48:05,958][44959] Updated weights for policy 1, policy_version 89600 (0.0008) [2023-10-12 23:48:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 183042048. Throughput: 0: 1639.8, 1: 1646.0. Samples: 45764820. Policy #0 lag: (min: 12.0, avg: 13.3, max: 38.0) [2023-10-12 23:48:06,443][43579] Avg episode reward: [(0, '288.000'), (1, '289.830')] [2023-10-12 23:48:06,660][44958] Updated weights for policy 0, policy_version 89160 (0.0008) [2023-10-12 23:48:07,034][44958] Updated weights for policy 0, policy_version 89170 (0.0007) [2023-10-12 23:48:07,402][44958] Updated weights for policy 0, policy_version 89180 (0.0007) [2023-10-12 23:48:10,337][44959] Updated weights for policy 1, policy_version 89610 (0.0008) [2023-10-12 23:48:10,694][44959] Updated weights for policy 1, policy_version 89620 (0.0007) [2023-10-12 23:48:11,065][44959] Updated weights for policy 1, policy_version 89630 (0.0007) [2023-10-12 23:48:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183107584. Throughput: 0: 1641.1, 1: 1640.7. Samples: 45784690. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:11,443][43579] Avg episode reward: [(0, '285.490'), (1, '290.970')] [2023-10-12 23:48:11,590][44958] Updated weights for policy 0, policy_version 89190 (0.0009) [2023-10-12 23:48:11,963][44958] Updated weights for policy 0, policy_version 89200 (0.0009) [2023-10-12 23:48:12,332][44958] Updated weights for policy 0, policy_version 89210 (0.0010) [2023-10-12 23:48:15,161][44959] Updated weights for policy 1, policy_version 89640 (0.0009) [2023-10-12 23:48:15,530][44959] Updated weights for policy 1, policy_version 89650 (0.0008) [2023-10-12 23:48:15,895][44959] Updated weights for policy 1, policy_version 89660 (0.0007) [2023-10-12 23:48:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183173120. Throughput: 0: 1635.8, 1: 1639.2. Samples: 45803782. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:16,443][43579] Avg episode reward: [(0, '280.040'), (1, '289.690')] [2023-10-12 23:48:16,681][44958] Updated weights for policy 0, policy_version 89220 (0.0009) [2023-10-12 23:48:17,054][44958] Updated weights for policy 0, policy_version 89230 (0.0007) [2023-10-12 23:48:17,421][44958] Updated weights for policy 0, policy_version 89240 (0.0008) [2023-10-12 23:48:20,124][44959] Updated weights for policy 1, policy_version 89670 (0.0008) [2023-10-12 23:48:20,497][44959] Updated weights for policy 1, policy_version 89680 (0.0007) [2023-10-12 23:48:20,857][44959] Updated weights for policy 1, policy_version 89690 (0.0008) [2023-10-12 23:48:21,440][44958] Updated weights for policy 0, policy_version 89250 (0.0010) [2023-10-12 23:48:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183238656. Throughput: 0: 1639.8, 1: 1643.2. Samples: 45813860. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:21,443][43579] Avg episode reward: [(0, '278.970'), (1, '288.690')] [2023-10-12 23:48:21,819][44958] Updated weights for policy 0, policy_version 89260 (0.0011) [2023-10-12 23:48:22,198][44958] Updated weights for policy 0, policy_version 89270 (0.0009) [2023-10-12 23:48:22,569][44958] Updated weights for policy 0, policy_version 89280 (0.0008) [2023-10-12 23:48:25,105][44959] Updated weights for policy 1, policy_version 89700 (0.0009) [2023-10-12 23:48:25,468][44959] Updated weights for policy 1, policy_version 89710 (0.0009) [2023-10-12 23:48:25,834][44959] Updated weights for policy 1, policy_version 89720 (0.0009) [2023-10-12 23:48:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183304192. Throughput: 0: 1642.5, 1: 1638.3. Samples: 45834170. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:26,443][43579] Avg episode reward: [(0, '273.450'), (1, '288.280')] [2023-10-12 23:48:26,807][44958] Updated weights for policy 0, policy_version 89290 (0.0009) [2023-10-12 23:48:27,181][44958] Updated weights for policy 0, policy_version 89300 (0.0009) [2023-10-12 23:48:27,556][44958] Updated weights for policy 0, policy_version 89310 (0.0009) [2023-10-12 23:48:30,162][44959] Updated weights for policy 1, policy_version 89730 (0.0008) [2023-10-12 23:48:30,581][44959] Updated weights for policy 1, policy_version 89740 (0.0008) [2023-10-12 23:48:30,953][44959] Updated weights for policy 1, policy_version 89750 (0.0008) [2023-10-12 23:48:31,323][44959] Updated weights for policy 1, policy_version 89760 (0.0009) [2023-10-12 23:48:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183369728. Throughput: 0: 1641.9, 1: 1639.2. Samples: 45853162. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:31,443][43579] Avg episode reward: [(0, '276.440'), (1, '290.760')] [2023-10-12 23:48:31,740][44958] Updated weights for policy 0, policy_version 89320 (0.0011) [2023-10-12 23:48:32,106][44958] Updated weights for policy 0, policy_version 89330 (0.0007) [2023-10-12 23:48:32,491][44958] Updated weights for policy 0, policy_version 89340 (0.0009) [2023-10-12 23:48:35,275][44959] Updated weights for policy 1, policy_version 89770 (0.0007) [2023-10-12 23:48:35,644][44959] Updated weights for policy 1, policy_version 89780 (0.0009) [2023-10-12 23:48:36,024][44959] Updated weights for policy 1, policy_version 89790 (0.0008) [2023-10-12 23:48:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183435264. Throughput: 0: 1641.1, 1: 1638.8. Samples: 45862908. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:36,443][43579] Avg episode reward: [(0, '274.520'), (1, '289.520')] [2023-10-12 23:48:36,693][44958] Updated weights for policy 0, policy_version 89350 (0.0007) [2023-10-12 23:48:37,059][44958] Updated weights for policy 0, policy_version 89360 (0.0008) [2023-10-12 23:48:37,440][44958] Updated weights for policy 0, policy_version 89370 (0.0008) [2023-10-12 23:48:40,315][44959] Updated weights for policy 1, policy_version 89800 (0.0009) [2023-10-12 23:48:40,679][44959] Updated weights for policy 1, policy_version 89810 (0.0007) [2023-10-12 23:48:41,059][44959] Updated weights for policy 1, policy_version 89820 (0.0008) [2023-10-12 23:48:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 183500800. Throughput: 0: 1639.4, 1: 1639.2. Samples: 45883184. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:41,444][43579] Avg episode reward: [(0, '275.530'), (1, '286.990')] [2023-10-12 23:48:41,714][44958] Updated weights for policy 0, policy_version 89380 (0.0009) [2023-10-12 23:48:42,095][44958] Updated weights for policy 0, policy_version 89390 (0.0010) [2023-10-12 23:48:42,463][44958] Updated weights for policy 0, policy_version 89400 (0.0007) [2023-10-12 23:48:45,144][44959] Updated weights for policy 1, policy_version 89830 (0.0008) [2023-10-12 23:48:45,505][44959] Updated weights for policy 1, policy_version 89840 (0.0008) [2023-10-12 23:48:45,865][44959] Updated weights for policy 1, policy_version 89850 (0.0012) [2023-10-12 23:48:46,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 183566336. Throughput: 0: 1636.8, 1: 1641.4. Samples: 45902330. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:46,444][43579] Avg episode reward: [(0, '274.130'), (1, '287.190')] [2023-10-12 23:48:46,680][44958] Updated weights for policy 0, policy_version 89410 (0.0008) [2023-10-12 23:48:47,055][44958] Updated weights for policy 0, policy_version 89420 (0.0011) [2023-10-12 23:48:47,432][44958] Updated weights for policy 0, policy_version 89430 (0.0007) [2023-10-12 23:48:47,800][44958] Updated weights for policy 0, policy_version 89440 (0.0008) [2023-10-12 23:48:50,076][44959] Updated weights for policy 1, policy_version 89860 (0.0010) [2023-10-12 23:48:50,450][44959] Updated weights for policy 1, policy_version 89870 (0.0009) [2023-10-12 23:48:50,810][44959] Updated weights for policy 1, policy_version 89880 (0.0008) [2023-10-12 23:48:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183631872. Throughput: 0: 1637.2, 1: 1637.6. Samples: 45912186. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:51,443][43579] Avg episode reward: [(0, '276.460'), (1, '285.270')] [2023-10-12 23:48:51,795][44958] Updated weights for policy 0, policy_version 89450 (0.0010) [2023-10-12 23:48:52,152][44958] Updated weights for policy 0, policy_version 89460 (0.0009) [2023-10-12 23:48:52,530][44958] Updated weights for policy 0, policy_version 89470 (0.0009) [2023-10-12 23:48:55,112][44959] Updated weights for policy 1, policy_version 89890 (0.0008) [2023-10-12 23:48:55,482][44959] Updated weights for policy 1, policy_version 89900 (0.0008) [2023-10-12 23:48:55,856][44959] Updated weights for policy 1, policy_version 89910 (0.0009) [2023-10-12 23:48:56,214][44959] Updated weights for policy 1, policy_version 89920 (0.0007) [2023-10-12 23:48:56,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183697408. Throughput: 0: 1638.7, 1: 1641.4. Samples: 45932294. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:48:56,443][43579] Avg episode reward: [(0, '275.740'), (1, '283.700')] [2023-10-12 23:48:56,695][44958] Updated weights for policy 0, policy_version 89480 (0.0009) [2023-10-12 23:48:57,081][44958] Updated weights for policy 0, policy_version 89490 (0.0010) [2023-10-12 23:48:57,463][44958] Updated weights for policy 0, policy_version 89500 (0.0011) [2023-10-12 23:49:00,399][44959] Updated weights for policy 1, policy_version 89930 (0.0007) [2023-10-12 23:49:00,761][44959] Updated weights for policy 1, policy_version 89940 (0.0007) [2023-10-12 23:49:01,135][44959] Updated weights for policy 1, policy_version 89950 (0.0010) [2023-10-12 23:49:01,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183762944. Throughput: 0: 1636.6, 1: 1639.6. Samples: 45951210. Policy #0 lag: (min: 18.0, avg: 24.3, max: 50.0) [2023-10-12 23:49:01,443][43579] Avg episode reward: [(0, '268.500'), (1, '286.170')] [2023-10-12 23:49:01,601][44958] Updated weights for policy 0, policy_version 89510 (0.0009) [2023-10-12 23:49:01,968][44958] Updated weights for policy 0, policy_version 89520 (0.0007) [2023-10-12 23:49:02,342][44958] Updated weights for policy 0, policy_version 89530 (0.0008) [2023-10-12 23:49:05,378][44959] Updated weights for policy 1, policy_version 89960 (0.0007) [2023-10-12 23:49:05,743][44959] Updated weights for policy 1, policy_version 89970 (0.0010) [2023-10-12 23:49:06,117][44959] Updated weights for policy 1, policy_version 89980 (0.0010) [2023-10-12 23:49:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183828480. Throughput: 0: 1636.7, 1: 1635.9. Samples: 45961128. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:06,443][43579] Avg episode reward: [(0, '270.990'), (1, '283.780')] [2023-10-12 23:49:06,573][44958] Updated weights for policy 0, policy_version 89540 (0.0008) [2023-10-12 23:49:06,936][44958] Updated weights for policy 0, policy_version 89550 (0.0009) [2023-10-12 23:49:07,311][44958] Updated weights for policy 0, policy_version 89560 (0.0009) [2023-10-12 23:49:10,063][44959] Updated weights for policy 1, policy_version 89990 (0.0009) [2023-10-12 23:49:10,438][44959] Updated weights for policy 1, policy_version 90000 (0.0008) [2023-10-12 23:49:10,804][44959] Updated weights for policy 1, policy_version 90010 (0.0008) [2023-10-12 23:49:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183894016. Throughput: 0: 1630.8, 1: 1636.5. Samples: 45981196. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:11,443][43579] Avg episode reward: [(0, '262.170'), (1, '285.150')] [2023-10-12 23:49:11,555][44958] Updated weights for policy 0, policy_version 89570 (0.0008) [2023-10-12 23:49:11,941][44958] Updated weights for policy 0, policy_version 89580 (0.0008) [2023-10-12 23:49:12,306][44958] Updated weights for policy 0, policy_version 89590 (0.0008) [2023-10-12 23:49:12,675][44958] Updated weights for policy 0, policy_version 89600 (0.0009) [2023-10-12 23:49:14,946][44959] Updated weights for policy 1, policy_version 90020 (0.0007) [2023-10-12 23:49:15,340][44959] Updated weights for policy 1, policy_version 90030 (0.0007) [2023-10-12 23:49:15,701][44959] Updated weights for policy 1, policy_version 90040 (0.0009) [2023-10-12 23:49:16,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 183959552. Throughput: 0: 1631.2, 1: 1642.8. Samples: 46000496. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:16,444][43579] Avg episode reward: [(0, '264.650'), (1, '284.290')] [2023-10-12 23:49:16,934][44958] Updated weights for policy 0, policy_version 89610 (0.0010) [2023-10-12 23:49:17,313][44958] Updated weights for policy 0, policy_version 89620 (0.0011) [2023-10-12 23:49:17,698][44958] Updated weights for policy 0, policy_version 89630 (0.0009) [2023-10-12 23:49:19,895][44959] Updated weights for policy 1, policy_version 90050 (0.0007) [2023-10-12 23:49:20,271][44959] Updated weights for policy 1, policy_version 90060 (0.0008) [2023-10-12 23:49:20,638][44959] Updated weights for policy 1, policy_version 90070 (0.0008) [2023-10-12 23:49:21,010][44959] Updated weights for policy 1, policy_version 90080 (0.0009) [2023-10-12 23:49:21,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184025088. Throughput: 0: 1633.3, 1: 1645.5. Samples: 46010454. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:21,443][43579] Avg episode reward: [(0, '266.610'), (1, '286.420')] [2023-10-12 23:49:21,798][44958] Updated weights for policy 0, policy_version 89640 (0.0012) [2023-10-12 23:49:22,166][44958] Updated weights for policy 0, policy_version 89650 (0.0010) [2023-10-12 23:49:22,543][44958] Updated weights for policy 0, policy_version 89660 (0.0011) [2023-10-12 23:49:25,248][44959] Updated weights for policy 1, policy_version 90090 (0.0007) [2023-10-12 23:49:25,624][44959] Updated weights for policy 1, policy_version 90100 (0.0007) [2023-10-12 23:49:25,995][44959] Updated weights for policy 1, policy_version 90110 (0.0010) [2023-10-12 23:49:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184090624. Throughput: 0: 1632.3, 1: 1636.9. Samples: 46030300. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:26,443][43579] Avg episode reward: [(0, '268.960'), (1, '278.090')] [2023-10-12 23:49:26,808][44958] Updated weights for policy 0, policy_version 89670 (0.0010) [2023-10-12 23:49:27,188][44958] Updated weights for policy 0, policy_version 89680 (0.0010) [2023-10-12 23:49:27,563][44958] Updated weights for policy 0, policy_version 89690 (0.0010) [2023-10-12 23:49:30,175][44959] Updated weights for policy 1, policy_version 90120 (0.0009) [2023-10-12 23:49:30,548][44959] Updated weights for policy 1, policy_version 90130 (0.0009) [2023-10-12 23:49:30,929][44959] Updated weights for policy 1, policy_version 90140 (0.0009) [2023-10-12 23:49:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184156160. Throughput: 0: 1633.1, 1: 1638.0. Samples: 46049528. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:31,444][43579] Avg episode reward: [(0, '267.290'), (1, '266.580')] [2023-10-12 23:49:31,724][44958] Updated weights for policy 0, policy_version 89700 (0.0010) [2023-10-12 23:49:32,096][44958] Updated weights for policy 0, policy_version 89710 (0.0008) [2023-10-12 23:49:32,463][44958] Updated weights for policy 0, policy_version 89720 (0.0009) [2023-10-12 23:49:35,029][44959] Updated weights for policy 1, policy_version 90150 (0.0007) [2023-10-12 23:49:35,405][44959] Updated weights for policy 1, policy_version 90160 (0.0007) [2023-10-12 23:49:35,769][44959] Updated weights for policy 1, policy_version 90170 (0.0007) [2023-10-12 23:49:36,414][44958] Updated weights for policy 0, policy_version 89730 (0.0008) [2023-10-12 23:49:36,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184221696. Throughput: 0: 1636.3, 1: 1640.7. Samples: 46059654. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:36,444][43579] Avg episode reward: [(0, '269.370'), (1, '263.780')] [2023-10-12 23:49:36,780][44958] Updated weights for policy 0, policy_version 89740 (0.0007) [2023-10-12 23:49:37,148][44958] Updated weights for policy 0, policy_version 89750 (0.0007) [2023-10-12 23:49:37,520][44958] Updated weights for policy 0, policy_version 89760 (0.0009) [2023-10-12 23:49:40,004][44959] Updated weights for policy 1, policy_version 90180 (0.0007) [2023-10-12 23:49:40,380][44959] Updated weights for policy 1, policy_version 90190 (0.0007) [2023-10-12 23:49:40,755][44959] Updated weights for policy 1, policy_version 90200 (0.0008) [2023-10-12 23:49:41,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 184287232. Throughput: 0: 1634.4, 1: 1640.0. Samples: 46079642. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:41,443][43579] Avg episode reward: [(0, '278.410'), (1, '259.500')] [2023-10-12 23:49:41,711][44958] Updated weights for policy 0, policy_version 89770 (0.0009) [2023-10-12 23:49:42,079][44958] Updated weights for policy 0, policy_version 89780 (0.0007) [2023-10-12 23:49:42,459][44958] Updated weights for policy 0, policy_version 89790 (0.0011) [2023-10-12 23:49:44,881][44959] Updated weights for policy 1, policy_version 90210 (0.0008) [2023-10-12 23:49:45,263][44959] Updated weights for policy 1, policy_version 90220 (0.0007) [2023-10-12 23:49:45,624][44959] Updated weights for policy 1, policy_version 90230 (0.0007) [2023-10-12 23:49:45,982][44959] Updated weights for policy 1, policy_version 90240 (0.0007) [2023-10-12 23:49:46,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184352768. Throughput: 0: 1634.8, 1: 1644.0. Samples: 46098754. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:46,443][43579] Avg episode reward: [(0, '277.560'), (1, '261.890')] [2023-10-12 23:49:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000090240_92405760.pth... [2023-10-12 23:49:46,482][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000088704_90832896.pth [2023-10-12 23:49:46,762][44958] Updated weights for policy 0, policy_version 89800 (0.0011) [2023-10-12 23:49:47,147][44958] Updated weights for policy 0, policy_version 89810 (0.0011) [2023-10-12 23:49:47,522][44958] Updated weights for policy 0, policy_version 89820 (0.0009) [2023-10-12 23:49:47,665][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000089824_91979776.pth... [2023-10-12 23:49:47,704][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000088288_90406912.pth [2023-10-12 23:49:50,329][44959] Updated weights for policy 1, policy_version 90250 (0.0008) [2023-10-12 23:49:50,694][44959] Updated weights for policy 1, policy_version 90260 (0.0008) [2023-10-12 23:49:51,062][44959] Updated weights for policy 1, policy_version 90270 (0.0009) [2023-10-12 23:49:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184418304. Throughput: 0: 1631.9, 1: 1647.6. Samples: 46108708. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:51,443][43579] Avg episode reward: [(0, '281.060'), (1, '263.890')] [2023-10-12 23:49:51,937][44958] Updated weights for policy 0, policy_version 89830 (0.0007) [2023-10-12 23:49:52,315][44958] Updated weights for policy 0, policy_version 89840 (0.0008) [2023-10-12 23:49:52,676][44958] Updated weights for policy 0, policy_version 89850 (0.0010) [2023-10-12 23:49:55,250][44959] Updated weights for policy 1, policy_version 90280 (0.0008) [2023-10-12 23:49:55,627][44959] Updated weights for policy 1, policy_version 90290 (0.0009) [2023-10-12 23:49:55,990][44959] Updated weights for policy 1, policy_version 90300 (0.0009) [2023-10-12 23:49:56,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184483840. Throughput: 0: 1629.6, 1: 1645.2. Samples: 46128566. Policy #0 lag: (min: 30.0, avg: 43.5, max: 62.0) [2023-10-12 23:49:56,444][43579] Avg episode reward: [(0, '281.530'), (1, '275.170')] [2023-10-12 23:49:56,957][44958] Updated weights for policy 0, policy_version 89860 (0.0009) [2023-10-12 23:49:57,343][44958] Updated weights for policy 0, policy_version 89870 (0.0010) [2023-10-12 23:49:57,719][44958] Updated weights for policy 0, policy_version 89880 (0.0008) [2023-10-12 23:50:00,051][44959] Updated weights for policy 1, policy_version 90310 (0.0009) [2023-10-12 23:50:00,450][44959] Updated weights for policy 1, policy_version 90320 (0.0007) [2023-10-12 23:50:00,817][44959] Updated weights for policy 1, policy_version 90330 (0.0007) [2023-10-12 23:50:01,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 184549376. Throughput: 0: 1628.6, 1: 1640.9. Samples: 46147622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:01,444][43579] Avg episode reward: [(0, '284.790'), (1, '275.150')] [2023-10-12 23:50:01,741][44958] Updated weights for policy 0, policy_version 89890 (0.0009) [2023-10-12 23:50:02,112][44958] Updated weights for policy 0, policy_version 89900 (0.0009) [2023-10-12 23:50:02,476][44958] Updated weights for policy 0, policy_version 89910 (0.0010) [2023-10-12 23:50:02,845][44958] Updated weights for policy 0, policy_version 89920 (0.0011) [2023-10-12 23:50:05,156][44959] Updated weights for policy 1, policy_version 90340 (0.0007) [2023-10-12 23:50:05,533][44959] Updated weights for policy 1, policy_version 90350 (0.0008) [2023-10-12 23:50:05,903][44959] Updated weights for policy 1, policy_version 90360 (0.0009) [2023-10-12 23:50:06,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184614912. Throughput: 0: 1625.9, 1: 1640.5. Samples: 46157440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:06,443][43579] Avg episode reward: [(0, '284.480'), (1, '276.050')] [2023-10-12 23:50:07,183][44958] Updated weights for policy 0, policy_version 89930 (0.0010) [2023-10-12 23:50:07,557][44958] Updated weights for policy 0, policy_version 89940 (0.0010) [2023-10-12 23:50:07,928][44958] Updated weights for policy 0, policy_version 89950 (0.0008) [2023-10-12 23:50:10,028][44959] Updated weights for policy 1, policy_version 90370 (0.0009) [2023-10-12 23:50:10,389][44959] Updated weights for policy 1, policy_version 90380 (0.0011) [2023-10-12 23:50:10,760][44959] Updated weights for policy 1, policy_version 90390 (0.0010) [2023-10-12 23:50:11,136][44959] Updated weights for policy 1, policy_version 90400 (0.0008) [2023-10-12 23:50:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184680448. Throughput: 0: 1628.5, 1: 1646.0. Samples: 46177654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:11,443][43579] Avg episode reward: [(0, '278.770'), (1, '281.060')] [2023-10-12 23:50:12,163][44958] Updated weights for policy 0, policy_version 89960 (0.0007) [2023-10-12 23:50:12,531][44958] Updated weights for policy 0, policy_version 89970 (0.0007) [2023-10-12 23:50:12,899][44958] Updated weights for policy 0, policy_version 89980 (0.0009) [2023-10-12 23:50:15,230][44959] Updated weights for policy 1, policy_version 90410 (0.0007) [2023-10-12 23:50:15,602][44959] Updated weights for policy 1, policy_version 90420 (0.0008) [2023-10-12 23:50:15,972][44959] Updated weights for policy 1, policy_version 90430 (0.0009) [2023-10-12 23:50:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184745984. Throughput: 0: 1633.2, 1: 1643.4. Samples: 46196974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:16,443][43579] Avg episode reward: [(0, '280.140'), (1, '282.580')] [2023-10-12 23:50:16,869][44958] Updated weights for policy 0, policy_version 89990 (0.0008) [2023-10-12 23:50:17,239][44958] Updated weights for policy 0, policy_version 90000 (0.0009) [2023-10-12 23:50:17,619][44958] Updated weights for policy 0, policy_version 90010 (0.0009) [2023-10-12 23:50:20,129][44959] Updated weights for policy 1, policy_version 90440 (0.0009) [2023-10-12 23:50:20,491][44959] Updated weights for policy 1, policy_version 90450 (0.0009) [2023-10-12 23:50:20,852][44959] Updated weights for policy 1, policy_version 90460 (0.0010) [2023-10-12 23:50:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184811520. Throughput: 0: 1629.7, 1: 1642.3. Samples: 46206896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:21,444][43579] Avg episode reward: [(0, '276.740'), (1, '284.900')] [2023-10-12 23:50:21,822][44958] Updated weights for policy 0, policy_version 90020 (0.0010) [2023-10-12 23:50:22,192][44958] Updated weights for policy 0, policy_version 90030 (0.0008) [2023-10-12 23:50:22,566][44958] Updated weights for policy 0, policy_version 90040 (0.0011) [2023-10-12 23:50:25,103][44959] Updated weights for policy 1, policy_version 90470 (0.0008) [2023-10-12 23:50:25,470][44959] Updated weights for policy 1, policy_version 90480 (0.0007) [2023-10-12 23:50:25,846][44959] Updated weights for policy 1, policy_version 90490 (0.0008) [2023-10-12 23:50:26,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184877056. Throughput: 0: 1631.5, 1: 1645.1. Samples: 46227092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:26,444][43579] Avg episode reward: [(0, '276.210'), (1, '287.370')] [2023-10-12 23:50:26,788][44958] Updated weights for policy 0, policy_version 90050 (0.0009) [2023-10-12 23:50:27,155][44958] Updated weights for policy 0, policy_version 90060 (0.0009) [2023-10-12 23:50:27,534][44958] Updated weights for policy 0, policy_version 90070 (0.0009) [2023-10-12 23:50:27,901][44958] Updated weights for policy 0, policy_version 90080 (0.0009) [2023-10-12 23:50:29,823][44959] Updated weights for policy 1, policy_version 90500 (0.0007) [2023-10-12 23:50:30,198][44959] Updated weights for policy 1, policy_version 90510 (0.0010) [2023-10-12 23:50:30,567][44959] Updated weights for policy 1, policy_version 90520 (0.0011) [2023-10-12 23:50:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 184942592. Throughput: 0: 1640.8, 1: 1646.8. Samples: 46246700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:31,444][43579] Avg episode reward: [(0, '276.140'), (1, '286.840')] [2023-10-12 23:50:32,033][44958] Updated weights for policy 0, policy_version 90090 (0.0009) [2023-10-12 23:50:32,401][44958] Updated weights for policy 0, policy_version 90100 (0.0009) [2023-10-12 23:50:32,783][44958] Updated weights for policy 0, policy_version 90110 (0.0008) [2023-10-12 23:50:34,701][44959] Updated weights for policy 1, policy_version 90530 (0.0009) [2023-10-12 23:50:35,070][44959] Updated weights for policy 1, policy_version 90540 (0.0008) [2023-10-12 23:50:35,437][44959] Updated weights for policy 1, policy_version 90550 (0.0008) [2023-10-12 23:50:35,807][44959] Updated weights for policy 1, policy_version 90560 (0.0008) [2023-10-12 23:50:36,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185008128. Throughput: 0: 1640.1, 1: 1647.3. Samples: 46256642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:36,443][43579] Avg episode reward: [(0, '277.790'), (1, '287.540')] [2023-10-12 23:50:36,888][44958] Updated weights for policy 0, policy_version 90120 (0.0009) [2023-10-12 23:50:37,267][44958] Updated weights for policy 0, policy_version 90130 (0.0007) [2023-10-12 23:50:37,639][44958] Updated weights for policy 0, policy_version 90140 (0.0008) [2023-10-12 23:50:39,909][44959] Updated weights for policy 1, policy_version 90570 (0.0008) [2023-10-12 23:50:40,287][44959] Updated weights for policy 1, policy_version 90580 (0.0008) [2023-10-12 23:50:40,655][44959] Updated weights for policy 1, policy_version 90590 (0.0008) [2023-10-12 23:50:41,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 185073664. Throughput: 0: 1641.6, 1: 1641.9. Samples: 46276324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:41,444][43579] Avg episode reward: [(0, '284.260'), (1, '286.930')] [2023-10-12 23:50:42,076][44958] Updated weights for policy 0, policy_version 90150 (0.0010) [2023-10-12 23:50:42,457][44958] Updated weights for policy 0, policy_version 90160 (0.0009) [2023-10-12 23:50:42,836][44958] Updated weights for policy 0, policy_version 90170 (0.0010) [2023-10-12 23:50:44,814][44959] Updated weights for policy 1, policy_version 90600 (0.0008) [2023-10-12 23:50:45,189][44959] Updated weights for policy 1, policy_version 90610 (0.0008) [2023-10-12 23:50:45,570][44959] Updated weights for policy 1, policy_version 90620 (0.0008) [2023-10-12 23:50:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185139200. Throughput: 0: 1642.4, 1: 1648.9. Samples: 46295730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:46,444][43579] Avg episode reward: [(0, '280.650'), (1, '286.950')] [2023-10-12 23:50:46,949][44958] Updated weights for policy 0, policy_version 90180 (0.0009) [2023-10-12 23:50:47,324][44958] Updated weights for policy 0, policy_version 90190 (0.0007) [2023-10-12 23:50:47,691][44958] Updated weights for policy 0, policy_version 90200 (0.0009) [2023-10-12 23:50:49,901][44959] Updated weights for policy 1, policy_version 90630 (0.0009) [2023-10-12 23:50:50,266][44959] Updated weights for policy 1, policy_version 90640 (0.0010) [2023-10-12 23:50:50,638][44959] Updated weights for policy 1, policy_version 90650 (0.0009) [2023-10-12 23:50:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185204736. Throughput: 0: 1642.9, 1: 1645.3. Samples: 46305410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:51,443][43579] Avg episode reward: [(0, '276.470'), (1, '286.470')] [2023-10-12 23:50:51,854][44958] Updated weights for policy 0, policy_version 90210 (0.0009) [2023-10-12 23:50:52,235][44958] Updated weights for policy 0, policy_version 90220 (0.0009) [2023-10-12 23:50:52,606][44958] Updated weights for policy 0, policy_version 90230 (0.0010) [2023-10-12 23:50:52,974][44958] Updated weights for policy 0, policy_version 90240 (0.0009) [2023-10-12 23:50:54,640][44959] Updated weights for policy 1, policy_version 90660 (0.0010) [2023-10-12 23:50:55,006][44959] Updated weights for policy 1, policy_version 90670 (0.0010) [2023-10-12 23:50:55,374][44959] Updated weights for policy 1, policy_version 90680 (0.0008) [2023-10-12 23:50:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185270272. Throughput: 0: 1645.8, 1: 1637.8. Samples: 46325418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:50:56,443][43579] Avg episode reward: [(0, '276.820'), (1, '275.290')] [2023-10-12 23:50:57,030][44958] Updated weights for policy 0, policy_version 90250 (0.0008) [2023-10-12 23:50:57,404][44958] Updated weights for policy 0, policy_version 90260 (0.0009) [2023-10-12 23:50:57,780][44958] Updated weights for policy 0, policy_version 90270 (0.0008) [2023-10-12 23:50:59,336][44959] Updated weights for policy 1, policy_version 90690 (0.0008) [2023-10-12 23:50:59,701][44959] Updated weights for policy 1, policy_version 90700 (0.0008) [2023-10-12 23:51:00,077][44959] Updated weights for policy 1, policy_version 90710 (0.0009) [2023-10-12 23:51:00,453][44959] Updated weights for policy 1, policy_version 90720 (0.0010) [2023-10-12 23:51:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185335808. Throughput: 0: 1640.3, 1: 1651.7. Samples: 46345112. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:01,443][43579] Avg episode reward: [(0, '278.160'), (1, '276.270')] [2023-10-12 23:51:01,939][44958] Updated weights for policy 0, policy_version 90280 (0.0010) [2023-10-12 23:51:02,317][44958] Updated weights for policy 0, policy_version 90290 (0.0008) [2023-10-12 23:51:02,679][44958] Updated weights for policy 0, policy_version 90300 (0.0008) [2023-10-12 23:51:04,855][44959] Updated weights for policy 1, policy_version 90730 (0.0009) [2023-10-12 23:51:05,229][44959] Updated weights for policy 1, policy_version 90740 (0.0007) [2023-10-12 23:51:05,594][44959] Updated weights for policy 1, policy_version 90750 (0.0008) [2023-10-12 23:51:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185401344. Throughput: 0: 1642.7, 1: 1650.6. Samples: 46355092. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:06,443][43579] Avg episode reward: [(0, '272.750'), (1, '275.310')] [2023-10-12 23:51:06,700][44958] Updated weights for policy 0, policy_version 90310 (0.0009) [2023-10-12 23:51:07,071][44958] Updated weights for policy 0, policy_version 90320 (0.0008) [2023-10-12 23:51:07,449][44958] Updated weights for policy 0, policy_version 90330 (0.0010) [2023-10-12 23:51:09,640][44959] Updated weights for policy 1, policy_version 90760 (0.0008) [2023-10-12 23:51:10,007][44959] Updated weights for policy 1, policy_version 90770 (0.0007) [2023-10-12 23:51:10,376][44959] Updated weights for policy 1, policy_version 90780 (0.0008) [2023-10-12 23:51:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185466880. Throughput: 0: 1644.5, 1: 1640.1. Samples: 46374900. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:11,443][43579] Avg episode reward: [(0, '271.960'), (1, '273.140')] [2023-10-12 23:51:11,662][44958] Updated weights for policy 0, policy_version 90340 (0.0008) [2023-10-12 23:51:12,047][44958] Updated weights for policy 0, policy_version 90350 (0.0009) [2023-10-12 23:51:12,422][44958] Updated weights for policy 0, policy_version 90360 (0.0009) [2023-10-12 23:51:14,622][44959] Updated weights for policy 1, policy_version 90790 (0.0008) [2023-10-12 23:51:14,991][44959] Updated weights for policy 1, policy_version 90800 (0.0007) [2023-10-12 23:51:15,349][44959] Updated weights for policy 1, policy_version 90810 (0.0007) [2023-10-12 23:51:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185532416. Throughput: 0: 1643.2, 1: 1645.0. Samples: 46394668. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:16,444][43579] Avg episode reward: [(0, '267.660'), (1, '272.000')] [2023-10-12 23:51:16,467][44958] Updated weights for policy 0, policy_version 90370 (0.0008) [2023-10-12 23:51:16,831][44958] Updated weights for policy 0, policy_version 90380 (0.0009) [2023-10-12 23:51:17,211][44958] Updated weights for policy 0, policy_version 90390 (0.0009) [2023-10-12 23:51:17,572][44958] Updated weights for policy 0, policy_version 90400 (0.0010) [2023-10-12 23:51:19,518][44959] Updated weights for policy 1, policy_version 90820 (0.0008) [2023-10-12 23:51:19,884][44959] Updated weights for policy 1, policy_version 90830 (0.0007) [2023-10-12 23:51:20,256][44959] Updated weights for policy 1, policy_version 90840 (0.0008) [2023-10-12 23:51:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185597952. Throughput: 0: 1644.7, 1: 1648.5. Samples: 46404838. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:21,443][43579] Avg episode reward: [(0, '270.310'), (1, '270.900')] [2023-10-12 23:51:21,796][44958] Updated weights for policy 0, policy_version 90410 (0.0007) [2023-10-12 23:51:22,170][44958] Updated weights for policy 0, policy_version 90420 (0.0008) [2023-10-12 23:51:22,545][44958] Updated weights for policy 0, policy_version 90430 (0.0007) [2023-10-12 23:51:24,454][44959] Updated weights for policy 1, policy_version 90850 (0.0008) [2023-10-12 23:51:24,833][44959] Updated weights for policy 1, policy_version 90860 (0.0009) [2023-10-12 23:51:25,207][44959] Updated weights for policy 1, policy_version 90870 (0.0008) [2023-10-12 23:51:25,567][44959] Updated weights for policy 1, policy_version 90880 (0.0007) [2023-10-12 23:51:26,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185663488. Throughput: 0: 1649.6, 1: 1640.8. Samples: 46424390. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:26,443][43579] Avg episode reward: [(0, '272.070'), (1, '282.730')] [2023-10-12 23:51:26,675][44958] Updated weights for policy 0, policy_version 90440 (0.0008) [2023-10-12 23:51:27,047][44958] Updated weights for policy 0, policy_version 90450 (0.0009) [2023-10-12 23:51:27,421][44958] Updated weights for policy 0, policy_version 90460 (0.0009) [2023-10-12 23:51:29,995][44959] Updated weights for policy 1, policy_version 90890 (0.0010) [2023-10-12 23:51:30,358][44959] Updated weights for policy 1, policy_version 90900 (0.0009) [2023-10-12 23:51:30,737][44959] Updated weights for policy 1, policy_version 90910 (0.0007) [2023-10-12 23:51:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 185729024. Throughput: 0: 1644.7, 1: 1636.5. Samples: 46443386. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:31,443][43579] Avg episode reward: [(0, '275.560'), (1, '279.450')] [2023-10-12 23:51:31,550][44958] Updated weights for policy 0, policy_version 90470 (0.0007) [2023-10-12 23:51:31,925][44958] Updated weights for policy 0, policy_version 90480 (0.0008) [2023-10-12 23:51:32,305][44958] Updated weights for policy 0, policy_version 90490 (0.0008) [2023-10-12 23:51:34,706][44959] Updated weights for policy 1, policy_version 90920 (0.0007) [2023-10-12 23:51:35,078][44959] Updated weights for policy 1, policy_version 90930 (0.0008) [2023-10-12 23:51:35,440][44959] Updated weights for policy 1, policy_version 90940 (0.0008) [2023-10-12 23:51:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 185794560. Throughput: 0: 1649.0, 1: 1646.2. Samples: 46453694. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:36,443][43579] Avg episode reward: [(0, '280.560'), (1, '282.010')] [2023-10-12 23:51:36,586][44958] Updated weights for policy 0, policy_version 90500 (0.0010) [2023-10-12 23:51:36,965][44958] Updated weights for policy 0, policy_version 90510 (0.0007) [2023-10-12 23:51:37,333][44958] Updated weights for policy 0, policy_version 90520 (0.0007) [2023-10-12 23:51:39,617][44959] Updated weights for policy 1, policy_version 90950 (0.0009) [2023-10-12 23:51:39,984][44959] Updated weights for policy 1, policy_version 90960 (0.0008) [2023-10-12 23:51:40,357][44959] Updated weights for policy 1, policy_version 90970 (0.0009) [2023-10-12 23:51:41,287][44958] Updated weights for policy 0, policy_version 90530 (0.0008) [2023-10-12 23:51:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 185860096. Throughput: 0: 1649.7, 1: 1640.4. Samples: 46473476. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:41,444][43579] Avg episode reward: [(0, '284.700'), (1, '286.560')] [2023-10-12 23:51:41,654][44958] Updated weights for policy 0, policy_version 90540 (0.0008) [2023-10-12 23:51:42,024][44958] Updated weights for policy 0, policy_version 90550 (0.0008) [2023-10-12 23:51:42,402][44958] Updated weights for policy 0, policy_version 90560 (0.0008) [2023-10-12 23:51:44,607][44959] Updated weights for policy 1, policy_version 90980 (0.0011) [2023-10-12 23:51:44,989][44959] Updated weights for policy 1, policy_version 90990 (0.0007) [2023-10-12 23:51:45,361][44959] Updated weights for policy 1, policy_version 91000 (0.0009) [2023-10-12 23:51:46,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 185925632. Throughput: 0: 1654.3, 1: 1635.2. Samples: 46493136. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:46,443][43579] Avg episode reward: [(0, '286.450'), (1, '282.770')] [2023-10-12 23:51:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000091008_93192192.pth... [2023-10-12 23:51:46,482][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000089472_91619328.pth [2023-10-12 23:51:46,602][44958] Updated weights for policy 0, policy_version 90570 (0.0008) [2023-10-12 23:51:46,981][44958] Updated weights for policy 0, policy_version 90580 (0.0007) [2023-10-12 23:51:47,356][44958] Updated weights for policy 0, policy_version 90590 (0.0007) [2023-10-12 23:51:47,423][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000090592_92766208.pth... [2023-10-12 23:51:47,463][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000089056_91193344.pth [2023-10-12 23:51:49,552][44959] Updated weights for policy 1, policy_version 91010 (0.0009) [2023-10-12 23:51:49,918][44959] Updated weights for policy 1, policy_version 91020 (0.0010) [2023-10-12 23:51:50,276][44959] Updated weights for policy 1, policy_version 91030 (0.0009) [2023-10-12 23:51:50,641][44959] Updated weights for policy 1, policy_version 91040 (0.0011) [2023-10-12 23:51:51,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 185991168. Throughput: 0: 1653.9, 1: 1637.5. Samples: 46503208. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:51,444][43579] Avg episode reward: [(0, '286.870'), (1, '276.590')] [2023-10-12 23:51:51,540][44958] Updated weights for policy 0, policy_version 90600 (0.0008) [2023-10-12 23:51:51,918][44958] Updated weights for policy 0, policy_version 90610 (0.0007) [2023-10-12 23:51:52,285][44958] Updated weights for policy 0, policy_version 90620 (0.0009) [2023-10-12 23:51:54,888][44959] Updated weights for policy 1, policy_version 91050 (0.0007) [2023-10-12 23:51:55,259][44959] Updated weights for policy 1, policy_version 91060 (0.0007) [2023-10-12 23:51:55,626][44959] Updated weights for policy 1, policy_version 91070 (0.0010) [2023-10-12 23:51:56,407][44958] Updated weights for policy 0, policy_version 90630 (0.0010) [2023-10-12 23:51:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186056704. Throughput: 0: 1653.4, 1: 1640.8. Samples: 46523140. Policy #0 lag: (min: 26.0, avg: 37.3, max: 58.0) [2023-10-12 23:51:56,443][43579] Avg episode reward: [(0, '287.220'), (1, '275.900')] [2023-10-12 23:51:56,774][44958] Updated weights for policy 0, policy_version 90640 (0.0007) [2023-10-12 23:51:57,142][44958] Updated weights for policy 0, policy_version 90650 (0.0009) [2023-10-12 23:51:59,665][44959] Updated weights for policy 1, policy_version 91080 (0.0008) [2023-10-12 23:52:00,038][44959] Updated weights for policy 1, policy_version 91090 (0.0009) [2023-10-12 23:52:00,408][44959] Updated weights for policy 1, policy_version 91100 (0.0008) [2023-10-12 23:52:01,355][44958] Updated weights for policy 0, policy_version 90660 (0.0008) [2023-10-12 23:52:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186122240. Throughput: 0: 1647.6, 1: 1643.0. Samples: 46542744. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:01,444][43579] Avg episode reward: [(0, '285.260'), (1, '274.000')] [2023-10-12 23:52:01,728][44958] Updated weights for policy 0, policy_version 90670 (0.0008) [2023-10-12 23:52:02,107][44958] Updated weights for policy 0, policy_version 90680 (0.0009) [2023-10-12 23:52:04,393][44959] Updated weights for policy 1, policy_version 91110 (0.0008) [2023-10-12 23:52:04,765][44959] Updated weights for policy 1, policy_version 91120 (0.0009) [2023-10-12 23:52:05,138][44959] Updated weights for policy 1, policy_version 91130 (0.0009) [2023-10-12 23:52:06,120][44958] Updated weights for policy 0, policy_version 90690 (0.0008) [2023-10-12 23:52:06,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186187776. Throughput: 0: 1650.5, 1: 1642.3. Samples: 46553012. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:06,443][43579] Avg episode reward: [(0, '283.470'), (1, '274.220')] [2023-10-12 23:52:06,486][44958] Updated weights for policy 0, policy_version 90700 (0.0008) [2023-10-12 23:52:06,857][44958] Updated weights for policy 0, policy_version 90710 (0.0010) [2023-10-12 23:52:07,228][44958] Updated weights for policy 0, policy_version 90720 (0.0008) [2023-10-12 23:52:09,464][44959] Updated weights for policy 1, policy_version 91140 (0.0009) [2023-10-12 23:52:09,828][44959] Updated weights for policy 1, policy_version 91150 (0.0007) [2023-10-12 23:52:10,201][44959] Updated weights for policy 1, policy_version 91160 (0.0009) [2023-10-12 23:52:11,276][44958] Updated weights for policy 0, policy_version 90730 (0.0010) [2023-10-12 23:52:11,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 186253312. Throughput: 0: 1651.2, 1: 1642.2. Samples: 46572594. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:11,444][43579] Avg episode reward: [(0, '279.480'), (1, '274.400')] [2023-10-12 23:52:11,651][44958] Updated weights for policy 0, policy_version 90740 (0.0009) [2023-10-12 23:52:12,019][44958] Updated weights for policy 0, policy_version 90750 (0.0007) [2023-10-12 23:52:14,452][44959] Updated weights for policy 1, policy_version 91170 (0.0010) [2023-10-12 23:52:14,882][44959] Updated weights for policy 1, policy_version 91180 (0.0007) [2023-10-12 23:52:15,261][44959] Updated weights for policy 1, policy_version 91190 (0.0009) [2023-10-12 23:52:15,631][44959] Updated weights for policy 1, policy_version 91200 (0.0009) [2023-10-12 23:52:16,147][44958] Updated weights for policy 0, policy_version 90760 (0.0008) [2023-10-12 23:52:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186318848. Throughput: 0: 1654.1, 1: 1645.9. Samples: 46591888. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:16,443][43579] Avg episode reward: [(0, '281.940'), (1, '274.680')] [2023-10-12 23:52:16,518][44958] Updated weights for policy 0, policy_version 90770 (0.0009) [2023-10-12 23:52:16,902][44958] Updated weights for policy 0, policy_version 90780 (0.0009) [2023-10-12 23:52:19,952][44959] Updated weights for policy 1, policy_version 91210 (0.0007) [2023-10-12 23:52:20,323][44959] Updated weights for policy 1, policy_version 91220 (0.0007) [2023-10-12 23:52:20,687][44959] Updated weights for policy 1, policy_version 91230 (0.0007) [2023-10-12 23:52:21,040][44958] Updated weights for policy 0, policy_version 90790 (0.0009) [2023-10-12 23:52:21,419][44958] Updated weights for policy 0, policy_version 90800 (0.0008) [2023-10-12 23:52:21,443][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186384384. Throughput: 0: 1660.5, 1: 1638.9. Samples: 46602168. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:21,443][43579] Avg episode reward: [(0, '278.110'), (1, '271.910')] [2023-10-12 23:52:21,785][44958] Updated weights for policy 0, policy_version 90810 (0.0008) [2023-10-12 23:52:24,746][44959] Updated weights for policy 1, policy_version 91240 (0.0007) [2023-10-12 23:52:25,113][44959] Updated weights for policy 1, policy_version 91250 (0.0010) [2023-10-12 23:52:25,476][44959] Updated weights for policy 1, policy_version 91260 (0.0008) [2023-10-12 23:52:25,882][44958] Updated weights for policy 0, policy_version 90820 (0.0009) [2023-10-12 23:52:26,262][44958] Updated weights for policy 0, policy_version 90830 (0.0007) [2023-10-12 23:52:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186449920. Throughput: 0: 1657.1, 1: 1645.0. Samples: 46622070. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:26,443][43579] Avg episode reward: [(0, '278.190'), (1, '271.080')] [2023-10-12 23:52:26,633][44958] Updated weights for policy 0, policy_version 90840 (0.0008) [2023-10-12 23:52:29,652][44959] Updated weights for policy 1, policy_version 91270 (0.0008) [2023-10-12 23:52:30,025][44959] Updated weights for policy 1, policy_version 91280 (0.0009) [2023-10-12 23:52:30,389][44959] Updated weights for policy 1, policy_version 91290 (0.0007) [2023-10-12 23:52:30,903][44958] Updated weights for policy 0, policy_version 90850 (0.0008) [2023-10-12 23:52:31,281][44958] Updated weights for policy 0, policy_version 90860 (0.0008) [2023-10-12 23:52:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186515456. Throughput: 0: 1649.2, 1: 1646.1. Samples: 46641424. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:31,443][43579] Avg episode reward: [(0, '278.910'), (1, '272.820')] [2023-10-12 23:52:31,656][44958] Updated weights for policy 0, policy_version 90870 (0.0008) [2023-10-12 23:52:32,017][44958] Updated weights for policy 0, policy_version 90880 (0.0007) [2023-10-12 23:52:34,455][44959] Updated weights for policy 1, policy_version 91300 (0.0007) [2023-10-12 23:52:34,822][44959] Updated weights for policy 1, policy_version 91310 (0.0010) [2023-10-12 23:52:35,195][44959] Updated weights for policy 1, policy_version 91320 (0.0007) [2023-10-12 23:52:36,225][44958] Updated weights for policy 0, policy_version 90890 (0.0009) [2023-10-12 23:52:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186580992. Throughput: 0: 1652.2, 1: 1648.8. Samples: 46651752. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:36,443][43579] Avg episode reward: [(0, '282.850'), (1, '272.070')] [2023-10-12 23:52:36,592][44958] Updated weights for policy 0, policy_version 90900 (0.0010) [2023-10-12 23:52:36,975][44958] Updated weights for policy 0, policy_version 90910 (0.0011) [2023-10-12 23:52:39,251][44959] Updated weights for policy 1, policy_version 91330 (0.0008) [2023-10-12 23:52:39,618][44959] Updated weights for policy 1, policy_version 91340 (0.0008) [2023-10-12 23:52:39,992][44959] Updated weights for policy 1, policy_version 91350 (0.0010) [2023-10-12 23:52:40,350][44959] Updated weights for policy 1, policy_version 91360 (0.0011) [2023-10-12 23:52:41,034][44958] Updated weights for policy 0, policy_version 90920 (0.0009) [2023-10-12 23:52:41,410][44958] Updated weights for policy 0, policy_version 90930 (0.0007) [2023-10-12 23:52:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186646528. Throughput: 0: 1649.8, 1: 1641.6. Samples: 46671250. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:41,443][43579] Avg episode reward: [(0, '282.440'), (1, '273.280')] [2023-10-12 23:52:41,787][44958] Updated weights for policy 0, policy_version 90940 (0.0009) [2023-10-12 23:52:44,307][44959] Updated weights for policy 1, policy_version 91370 (0.0008) [2023-10-12 23:52:44,672][44959] Updated weights for policy 1, policy_version 91380 (0.0008) [2023-10-12 23:52:45,040][44959] Updated weights for policy 1, policy_version 91390 (0.0009) [2023-10-12 23:52:46,072][44958] Updated weights for policy 0, policy_version 90950 (0.0010) [2023-10-12 23:52:46,434][44958] Updated weights for policy 0, policy_version 90960 (0.0008) [2023-10-12 23:52:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186712064. Throughput: 0: 1644.8, 1: 1649.5. Samples: 46690986. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:46,443][43579] Avg episode reward: [(0, '283.310'), (1, '273.700')] [2023-10-12 23:52:46,812][44958] Updated weights for policy 0, policy_version 90970 (0.0007) [2023-10-12 23:52:49,295][44959] Updated weights for policy 1, policy_version 91400 (0.0008) [2023-10-12 23:52:49,660][44959] Updated weights for policy 1, policy_version 91410 (0.0010) [2023-10-12 23:52:50,030][44959] Updated weights for policy 1, policy_version 91420 (0.0010) [2023-10-12 23:52:50,911][44958] Updated weights for policy 0, policy_version 90980 (0.0009) [2023-10-12 23:52:51,292][44958] Updated weights for policy 0, policy_version 90990 (0.0010) [2023-10-12 23:52:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 186777600. Throughput: 0: 1644.5, 1: 1645.4. Samples: 46701058. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:51,443][43579] Avg episode reward: [(0, '283.650'), (1, '279.240')] [2023-10-12 23:52:51,651][44958] Updated weights for policy 0, policy_version 91000 (0.0008) [2023-10-12 23:52:54,209][44959] Updated weights for policy 1, policy_version 91430 (0.0008) [2023-10-12 23:52:54,582][44959] Updated weights for policy 1, policy_version 91440 (0.0007) [2023-10-12 23:52:54,944][44959] Updated weights for policy 1, policy_version 91450 (0.0007) [2023-10-12 23:52:55,938][44958] Updated weights for policy 0, policy_version 91010 (0.0008) [2023-10-12 23:52:56,303][44958] Updated weights for policy 0, policy_version 91020 (0.0010) [2023-10-12 23:52:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186843136. Throughput: 0: 1645.3, 1: 1642.5. Samples: 46720540. Policy #0 lag: (min: 20.0, avg: 27.7, max: 52.0) [2023-10-12 23:52:56,443][43579] Avg episode reward: [(0, '286.950'), (1, '276.000')] [2023-10-12 23:52:56,671][44958] Updated weights for policy 0, policy_version 91030 (0.0007) [2023-10-12 23:52:57,040][44958] Updated weights for policy 0, policy_version 91040 (0.0010) [2023-10-12 23:52:59,162][44959] Updated weights for policy 1, policy_version 91460 (0.0008) [2023-10-12 23:52:59,527][44959] Updated weights for policy 1, policy_version 91470 (0.0007) [2023-10-12 23:52:59,893][44959] Updated weights for policy 1, policy_version 91480 (0.0009) [2023-10-12 23:53:01,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186908672. Throughput: 0: 1647.2, 1: 1655.3. Samples: 46740500. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:01,443][43579] Avg episode reward: [(0, '288.220'), (1, '274.830')] [2023-10-12 23:53:01,506][44958] Updated weights for policy 0, policy_version 91050 (0.0009) [2023-10-12 23:53:01,880][44958] Updated weights for policy 0, policy_version 91060 (0.0009) [2023-10-12 23:53:02,256][44958] Updated weights for policy 0, policy_version 91070 (0.0008) [2023-10-12 23:53:03,914][44959] Updated weights for policy 1, policy_version 91490 (0.0009) [2023-10-12 23:53:04,330][44959] Updated weights for policy 1, policy_version 91500 (0.0011) [2023-10-12 23:53:04,696][44959] Updated weights for policy 1, policy_version 91510 (0.0011) [2023-10-12 23:53:05,064][44959] Updated weights for policy 1, policy_version 91520 (0.0010) [2023-10-12 23:53:06,191][44958] Updated weights for policy 0, policy_version 91080 (0.0009) [2023-10-12 23:53:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 186974208. Throughput: 0: 1637.1, 1: 1655.3. Samples: 46750326. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:06,443][43579] Avg episode reward: [(0, '288.330'), (1, '276.770')] [2023-10-12 23:53:06,567][44958] Updated weights for policy 0, policy_version 91090 (0.0007) [2023-10-12 23:53:06,941][44958] Updated weights for policy 0, policy_version 91100 (0.0009) [2023-10-12 23:53:09,296][44959] Updated weights for policy 1, policy_version 91530 (0.0009) [2023-10-12 23:53:09,668][44959] Updated weights for policy 1, policy_version 91540 (0.0007) [2023-10-12 23:53:10,035][44959] Updated weights for policy 1, policy_version 91550 (0.0009) [2023-10-12 23:53:11,089][44958] Updated weights for policy 0, policy_version 91110 (0.0009) [2023-10-12 23:53:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 187039744. Throughput: 0: 1635.8, 1: 1644.2. Samples: 46769670. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:11,443][43579] Avg episode reward: [(0, '289.290'), (1, '279.850')] [2023-10-12 23:53:11,468][44958] Updated weights for policy 0, policy_version 91120 (0.0007) [2023-10-12 23:53:11,835][44958] Updated weights for policy 0, policy_version 91130 (0.0008) [2023-10-12 23:53:14,148][44959] Updated weights for policy 1, policy_version 91560 (0.0009) [2023-10-12 23:53:14,519][44959] Updated weights for policy 1, policy_version 91570 (0.0008) [2023-10-12 23:53:14,887][44959] Updated weights for policy 1, policy_version 91580 (0.0008) [2023-10-12 23:53:16,123][44958] Updated weights for policy 0, policy_version 91140 (0.0008) [2023-10-12 23:53:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187105280. Throughput: 0: 1632.9, 1: 1655.8. Samples: 46789414. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:16,443][43579] Avg episode reward: [(0, '288.000'), (1, '275.070')] [2023-10-12 23:53:16,505][44958] Updated weights for policy 0, policy_version 91150 (0.0010) [2023-10-12 23:53:16,874][44958] Updated weights for policy 0, policy_version 91160 (0.0009) [2023-10-12 23:53:19,056][44959] Updated weights for policy 1, policy_version 91590 (0.0008) [2023-10-12 23:53:19,429][44959] Updated weights for policy 1, policy_version 91600 (0.0007) [2023-10-12 23:53:19,796][44959] Updated weights for policy 1, policy_version 91610 (0.0008) [2023-10-12 23:53:21,136][44958] Updated weights for policy 0, policy_version 91170 (0.0009) [2023-10-12 23:53:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187170816. Throughput: 0: 1632.1, 1: 1648.8. Samples: 46799392. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:21,443][43579] Avg episode reward: [(0, '288.550'), (1, '281.040')] [2023-10-12 23:53:21,500][44958] Updated weights for policy 0, policy_version 91180 (0.0008) [2023-10-12 23:53:21,865][44958] Updated weights for policy 0, policy_version 91190 (0.0007) [2023-10-12 23:53:22,233][44958] Updated weights for policy 0, policy_version 91200 (0.0009) [2023-10-12 23:53:23,878][44959] Updated weights for policy 1, policy_version 91620 (0.0008) [2023-10-12 23:53:24,249][44959] Updated weights for policy 1, policy_version 91630 (0.0009) [2023-10-12 23:53:24,623][44959] Updated weights for policy 1, policy_version 91640 (0.0007) [2023-10-12 23:53:26,361][44958] Updated weights for policy 0, policy_version 91210 (0.0007) [2023-10-12 23:53:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187236352. Throughput: 0: 1635.3, 1: 1642.9. Samples: 46818770. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:26,443][43579] Avg episode reward: [(0, '286.810'), (1, '282.220')] [2023-10-12 23:53:26,731][44958] Updated weights for policy 0, policy_version 91220 (0.0008) [2023-10-12 23:53:27,101][44958] Updated weights for policy 0, policy_version 91230 (0.0007) [2023-10-12 23:53:28,712][44959] Updated weights for policy 1, policy_version 91650 (0.0008) [2023-10-12 23:53:29,074][44959] Updated weights for policy 1, policy_version 91660 (0.0007) [2023-10-12 23:53:29,445][44959] Updated weights for policy 1, policy_version 91670 (0.0008) [2023-10-12 23:53:29,815][44959] Updated weights for policy 1, policy_version 91680 (0.0008) [2023-10-12 23:53:31,304][44958] Updated weights for policy 0, policy_version 91240 (0.0009) [2023-10-12 23:53:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187301888. Throughput: 0: 1637.0, 1: 1651.9. Samples: 46838986. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:31,443][43579] Avg episode reward: [(0, '286.430'), (1, '283.620')] [2023-10-12 23:53:31,673][44958] Updated weights for policy 0, policy_version 91250 (0.0008) [2023-10-12 23:53:32,042][44958] Updated weights for policy 0, policy_version 91260 (0.0007) [2023-10-12 23:53:34,018][44959] Updated weights for policy 1, policy_version 91690 (0.0010) [2023-10-12 23:53:34,388][44959] Updated weights for policy 1, policy_version 91700 (0.0007) [2023-10-12 23:53:34,762][44959] Updated weights for policy 1, policy_version 91710 (0.0007) [2023-10-12 23:53:36,069][44958] Updated weights for policy 0, policy_version 91270 (0.0010) [2023-10-12 23:53:36,437][44958] Updated weights for policy 0, policy_version 91280 (0.0008) [2023-10-12 23:53:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187367424. Throughput: 0: 1637.9, 1: 1647.8. Samples: 46848916. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:36,443][43579] Avg episode reward: [(0, '286.540'), (1, '282.850')] [2023-10-12 23:53:36,809][44958] Updated weights for policy 0, policy_version 91290 (0.0010) [2023-10-12 23:53:38,938][44959] Updated weights for policy 1, policy_version 91720 (0.0008) [2023-10-12 23:53:39,300][44959] Updated weights for policy 1, policy_version 91730 (0.0008) [2023-10-12 23:53:39,661][44959] Updated weights for policy 1, policy_version 91740 (0.0007) [2023-10-12 23:53:40,993][44958] Updated weights for policy 0, policy_version 91300 (0.0010) [2023-10-12 23:53:41,362][44958] Updated weights for policy 0, policy_version 91310 (0.0009) [2023-10-12 23:53:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187432960. Throughput: 0: 1637.1, 1: 1648.9. Samples: 46868410. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:41,443][43579] Avg episode reward: [(0, '285.170'), (1, '283.010')] [2023-10-12 23:53:41,734][44958] Updated weights for policy 0, policy_version 91320 (0.0009) [2023-10-12 23:53:43,815][44959] Updated weights for policy 1, policy_version 91750 (0.0011) [2023-10-12 23:53:44,181][44959] Updated weights for policy 1, policy_version 91760 (0.0010) [2023-10-12 23:53:44,552][44959] Updated weights for policy 1, policy_version 91770 (0.0011) [2023-10-12 23:53:45,897][44958] Updated weights for policy 0, policy_version 91330 (0.0010) [2023-10-12 23:53:46,286][44958] Updated weights for policy 0, policy_version 91340 (0.0007) [2023-10-12 23:53:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187498496. Throughput: 0: 1630.5, 1: 1651.8. Samples: 46888202. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:46,443][43579] Avg episode reward: [(0, '285.100'), (1, '286.810')] [2023-10-12 23:53:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000091776_93978624.pth... [2023-10-12 23:53:46,486][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000090240_92405760.pth [2023-10-12 23:53:46,661][44958] Updated weights for policy 0, policy_version 91350 (0.0010) [2023-10-12 23:53:47,028][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000091360_93552640.pth... [2023-10-12 23:53:47,031][44958] Updated weights for policy 0, policy_version 91360 (0.0011) [2023-10-12 23:53:47,065][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000089824_91979776.pth [2023-10-12 23:53:48,785][44959] Updated weights for policy 1, policy_version 91780 (0.0010) [2023-10-12 23:53:49,189][44959] Updated weights for policy 1, policy_version 91790 (0.0008) [2023-10-12 23:53:49,561][44959] Updated weights for policy 1, policy_version 91800 (0.0009) [2023-10-12 23:53:51,216][44958] Updated weights for policy 0, policy_version 91370 (0.0008) [2023-10-12 23:53:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187564032. Throughput: 0: 1639.3, 1: 1641.8. Samples: 46897978. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:51,443][43579] Avg episode reward: [(0, '284.940'), (1, '286.210')] [2023-10-12 23:53:51,574][44958] Updated weights for policy 0, policy_version 91380 (0.0008) [2023-10-12 23:53:51,964][44958] Updated weights for policy 0, policy_version 91390 (0.0009) [2023-10-12 23:53:53,766][44959] Updated weights for policy 1, policy_version 91810 (0.0009) [2023-10-12 23:53:54,132][44959] Updated weights for policy 1, policy_version 91820 (0.0010) [2023-10-12 23:53:54,503][44959] Updated weights for policy 1, policy_version 91830 (0.0009) [2023-10-12 23:53:54,861][44959] Updated weights for policy 1, policy_version 91840 (0.0009) [2023-10-12 23:53:55,978][44958] Updated weights for policy 0, policy_version 91400 (0.0008) [2023-10-12 23:53:56,349][44958] Updated weights for policy 0, policy_version 91410 (0.0008) [2023-10-12 23:53:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187629568. Throughput: 0: 1641.1, 1: 1637.6. Samples: 46917210. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-12 23:53:56,443][43579] Avg episode reward: [(0, '279.610'), (1, '287.040')] [2023-10-12 23:53:56,722][44958] Updated weights for policy 0, policy_version 91420 (0.0009) [2023-10-12 23:53:59,309][44959] Updated weights for policy 1, policy_version 91850 (0.0008) [2023-10-12 23:53:59,686][44959] Updated weights for policy 1, policy_version 91860 (0.0007) [2023-10-12 23:54:00,049][44959] Updated weights for policy 1, policy_version 91870 (0.0007) [2023-10-12 23:54:00,779][44958] Updated weights for policy 0, policy_version 91430 (0.0008) [2023-10-12 23:54:01,162][44958] Updated weights for policy 0, policy_version 91440 (0.0008) [2023-10-12 23:54:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187695104. Throughput: 0: 1638.8, 1: 1642.2. Samples: 46937062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:01,443][43579] Avg episode reward: [(0, '274.010'), (1, '284.620')] [2023-10-12 23:54:01,536][44958] Updated weights for policy 0, policy_version 91450 (0.0009) [2023-10-12 23:54:04,079][44959] Updated weights for policy 1, policy_version 91880 (0.0007) [2023-10-12 23:54:04,439][44959] Updated weights for policy 1, policy_version 91890 (0.0008) [2023-10-12 23:54:04,817][44959] Updated weights for policy 1, policy_version 91900 (0.0009) [2023-10-12 23:54:05,793][44958] Updated weights for policy 0, policy_version 91460 (0.0009) [2023-10-12 23:54:06,165][44958] Updated weights for policy 0, policy_version 91470 (0.0008) [2023-10-12 23:54:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187760640. Throughput: 0: 1650.9, 1: 1643.4. Samples: 46947636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:06,443][43579] Avg episode reward: [(0, '273.790'), (1, '286.020')] [2023-10-12 23:54:06,549][44958] Updated weights for policy 0, policy_version 91480 (0.0009) [2023-10-12 23:54:08,938][44959] Updated weights for policy 1, policy_version 91910 (0.0008) [2023-10-12 23:54:09,295][44959] Updated weights for policy 1, policy_version 91920 (0.0008) [2023-10-12 23:54:09,660][44959] Updated weights for policy 1, policy_version 91930 (0.0011) [2023-10-12 23:54:10,766][44958] Updated weights for policy 0, policy_version 91490 (0.0008) [2023-10-12 23:54:11,143][44958] Updated weights for policy 0, policy_version 91500 (0.0008) [2023-10-12 23:54:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187826176. Throughput: 0: 1648.7, 1: 1644.7. Samples: 46966974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:11,443][43579] Avg episode reward: [(0, '271.860'), (1, '286.170')] [2023-10-12 23:54:11,517][44958] Updated weights for policy 0, policy_version 91510 (0.0009) [2023-10-12 23:54:11,886][44958] Updated weights for policy 0, policy_version 91520 (0.0008) [2023-10-12 23:54:13,911][44959] Updated weights for policy 1, policy_version 91940 (0.0009) [2023-10-12 23:54:14,275][44959] Updated weights for policy 1, policy_version 91950 (0.0009) [2023-10-12 23:54:14,648][44959] Updated weights for policy 1, policy_version 91960 (0.0007) [2023-10-12 23:54:16,010][44958] Updated weights for policy 0, policy_version 91530 (0.0007) [2023-10-12 23:54:16,387][44958] Updated weights for policy 0, policy_version 91540 (0.0007) [2023-10-12 23:54:16,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 187891712. Throughput: 0: 1643.4, 1: 1639.4. Samples: 46986710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:16,443][43579] Avg episode reward: [(0, '276.110'), (1, '280.980')] [2023-10-12 23:54:16,749][44958] Updated weights for policy 0, policy_version 91550 (0.0009) [2023-10-12 23:54:18,688][44959] Updated weights for policy 1, policy_version 91970 (0.0008) [2023-10-12 23:54:19,053][44959] Updated weights for policy 1, policy_version 91980 (0.0007) [2023-10-12 23:54:19,421][44959] Updated weights for policy 1, policy_version 91990 (0.0009) [2023-10-12 23:54:19,792][44959] Updated weights for policy 1, policy_version 92000 (0.0009) [2023-10-12 23:54:21,028][44958] Updated weights for policy 0, policy_version 91560 (0.0008) [2023-10-12 23:54:21,396][44958] Updated weights for policy 0, policy_version 91570 (0.0009) [2023-10-12 23:54:21,443][43579] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 187957248. Throughput: 0: 1652.7, 1: 1636.9. Samples: 46996950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:21,444][43579] Avg episode reward: [(0, '278.980'), (1, '278.760')] [2023-10-12 23:54:21,769][44958] Updated weights for policy 0, policy_version 91580 (0.0010) [2023-10-12 23:54:23,906][44959] Updated weights for policy 1, policy_version 92010 (0.0010) [2023-10-12 23:54:24,270][44959] Updated weights for policy 1, policy_version 92020 (0.0009) [2023-10-12 23:54:24,634][44959] Updated weights for policy 1, policy_version 92030 (0.0009) [2023-10-12 23:54:26,027][44958] Updated weights for policy 0, policy_version 91590 (0.0010) [2023-10-12 23:54:26,407][44958] Updated weights for policy 0, policy_version 91600 (0.0009) [2023-10-12 23:54:26,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188022784. Throughput: 0: 1652.0, 1: 1638.2. Samples: 47016472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:26,443][43579] Avg episode reward: [(0, '281.660'), (1, '278.210')] [2023-10-12 23:54:26,786][44958] Updated weights for policy 0, policy_version 91610 (0.0010) [2023-10-12 23:54:28,828][44959] Updated weights for policy 1, policy_version 92040 (0.0008) [2023-10-12 23:54:29,196][44959] Updated weights for policy 1, policy_version 92050 (0.0007) [2023-10-12 23:54:29,574][44959] Updated weights for policy 1, policy_version 92060 (0.0008) [2023-10-12 23:54:30,859][44958] Updated weights for policy 0, policy_version 91620 (0.0009) [2023-10-12 23:54:31,229][44958] Updated weights for policy 0, policy_version 91630 (0.0009) [2023-10-12 23:54:31,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188088320. Throughput: 0: 1652.1, 1: 1640.9. Samples: 47036384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:31,443][43579] Avg episode reward: [(0, '283.300'), (1, '278.920')] [2023-10-12 23:54:31,598][44958] Updated weights for policy 0, policy_version 91640 (0.0007) [2023-10-12 23:54:33,848][44959] Updated weights for policy 1, policy_version 92070 (0.0007) [2023-10-12 23:54:34,233][44959] Updated weights for policy 1, policy_version 92080 (0.0007) [2023-10-12 23:54:34,604][44959] Updated weights for policy 1, policy_version 92090 (0.0007) [2023-10-12 23:54:35,653][44958] Updated weights for policy 0, policy_version 91650 (0.0008) [2023-10-12 23:54:36,047][44958] Updated weights for policy 0, policy_version 91660 (0.0010) [2023-10-12 23:54:36,411][44958] Updated weights for policy 0, policy_version 91670 (0.0007) [2023-10-12 23:54:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188153856. Throughput: 0: 1657.6, 1: 1639.6. Samples: 47046350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:36,443][43579] Avg episode reward: [(0, '285.400'), (1, '280.290')] [2023-10-12 23:54:36,785][44958] Updated weights for policy 0, policy_version 91680 (0.0008) [2023-10-12 23:54:38,678][44959] Updated weights for policy 1, policy_version 92100 (0.0007) [2023-10-12 23:54:39,055][44959] Updated weights for policy 1, policy_version 92110 (0.0007) [2023-10-12 23:54:39,417][44959] Updated weights for policy 1, policy_version 92120 (0.0009) [2023-10-12 23:54:40,941][44958] Updated weights for policy 0, policy_version 91690 (0.0011) [2023-10-12 23:54:41,319][44958] Updated weights for policy 0, policy_version 91700 (0.0009) [2023-10-12 23:54:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188219392. Throughput: 0: 1653.3, 1: 1651.4. Samples: 47065922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:41,443][43579] Avg episode reward: [(0, '288.680'), (1, '280.090')] [2023-10-12 23:54:41,687][44958] Updated weights for policy 0, policy_version 91710 (0.0007) [2023-10-12 23:54:43,580][44959] Updated weights for policy 1, policy_version 92130 (0.0007) [2023-10-12 23:54:43,950][44959] Updated weights for policy 1, policy_version 92140 (0.0008) [2023-10-12 23:54:44,326][44959] Updated weights for policy 1, policy_version 92150 (0.0007) [2023-10-12 23:54:44,700][44959] Updated weights for policy 1, policy_version 92160 (0.0009) [2023-10-12 23:54:45,920][44958] Updated weights for policy 0, policy_version 91720 (0.0009) [2023-10-12 23:54:46,294][44958] Updated weights for policy 0, policy_version 91730 (0.0009) [2023-10-12 23:54:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188284928. Throughput: 0: 1649.3, 1: 1651.2. Samples: 47085588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:46,443][43579] Avg episode reward: [(0, '288.350'), (1, '282.590')] [2023-10-12 23:54:46,667][44958] Updated weights for policy 0, policy_version 91740 (0.0009) [2023-10-12 23:54:48,888][44959] Updated weights for policy 1, policy_version 92170 (0.0010) [2023-10-12 23:54:49,260][44959] Updated weights for policy 1, policy_version 92180 (0.0008) [2023-10-12 23:54:49,627][44959] Updated weights for policy 1, policy_version 92190 (0.0009) [2023-10-12 23:54:50,812][44958] Updated weights for policy 0, policy_version 91750 (0.0008) [2023-10-12 23:54:51,190][44958] Updated weights for policy 0, policy_version 91760 (0.0008) [2023-10-12 23:54:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188350464. Throughput: 0: 1649.4, 1: 1640.3. Samples: 47095670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:51,443][43579] Avg episode reward: [(0, '285.900'), (1, '284.200')] [2023-10-12 23:54:51,567][44958] Updated weights for policy 0, policy_version 91770 (0.0007) [2023-10-12 23:54:53,851][44959] Updated weights for policy 1, policy_version 92200 (0.0008) [2023-10-12 23:54:54,216][44959] Updated weights for policy 1, policy_version 92210 (0.0011) [2023-10-12 23:54:54,587][44959] Updated weights for policy 1, policy_version 92220 (0.0010) [2023-10-12 23:54:55,658][44958] Updated weights for policy 0, policy_version 91780 (0.0009) [2023-10-12 23:54:56,037][44958] Updated weights for policy 0, policy_version 91790 (0.0008) [2023-10-12 23:54:56,421][44958] Updated weights for policy 0, policy_version 91800 (0.0010) [2023-10-12 23:54:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188416000. Throughput: 0: 1650.2, 1: 1644.1. Samples: 47115216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:54:56,443][43579] Avg episode reward: [(0, '282.150'), (1, '283.850')] [2023-10-12 23:54:58,699][44959] Updated weights for policy 1, policy_version 92230 (0.0009) [2023-10-12 23:54:59,068][44959] Updated weights for policy 1, policy_version 92240 (0.0008) [2023-10-12 23:54:59,436][44959] Updated weights for policy 1, policy_version 92250 (0.0009) [2023-10-12 23:55:00,714][44958] Updated weights for policy 0, policy_version 91810 (0.0009) [2023-10-12 23:55:01,076][44958] Updated weights for policy 0, policy_version 91820 (0.0007) [2023-10-12 23:55:01,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 188481536. Throughput: 0: 1645.5, 1: 1648.2. Samples: 47134924. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:01,444][43579] Avg episode reward: [(0, '283.500'), (1, '278.200')] [2023-10-12 23:55:01,451][44958] Updated weights for policy 0, policy_version 91830 (0.0008) [2023-10-12 23:55:01,824][44958] Updated weights for policy 0, policy_version 91840 (0.0007) [2023-10-12 23:55:03,515][44959] Updated weights for policy 1, policy_version 92260 (0.0009) [2023-10-12 23:55:03,887][44959] Updated weights for policy 1, policy_version 92270 (0.0012) [2023-10-12 23:55:04,255][44959] Updated weights for policy 1, policy_version 92280 (0.0009) [2023-10-12 23:55:05,915][44958] Updated weights for policy 0, policy_version 91850 (0.0008) [2023-10-12 23:55:06,283][44958] Updated weights for policy 0, policy_version 91860 (0.0008) [2023-10-12 23:55:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188547072. Throughput: 0: 1644.5, 1: 1642.5. Samples: 47144864. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:06,443][43579] Avg episode reward: [(0, '277.980'), (1, '280.110')] [2023-10-12 23:55:06,655][44958] Updated weights for policy 0, policy_version 91870 (0.0009) [2023-10-12 23:55:08,268][44959] Updated weights for policy 1, policy_version 92290 (0.0009) [2023-10-12 23:55:08,636][44959] Updated weights for policy 1, policy_version 92300 (0.0008) [2023-10-12 23:55:08,994][44959] Updated weights for policy 1, policy_version 92310 (0.0007) [2023-10-12 23:55:09,370][44959] Updated weights for policy 1, policy_version 92320 (0.0008) [2023-10-12 23:55:10,849][44958] Updated weights for policy 0, policy_version 91880 (0.0010) [2023-10-12 23:55:11,213][44958] Updated weights for policy 0, policy_version 91890 (0.0009) [2023-10-12 23:55:11,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188612608. Throughput: 0: 1646.6, 1: 1653.8. Samples: 47164990. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:11,444][43579] Avg episode reward: [(0, '278.190'), (1, '273.200')] [2023-10-12 23:55:11,599][44958] Updated weights for policy 0, policy_version 91900 (0.0007) [2023-10-12 23:55:13,391][44959] Updated weights for policy 1, policy_version 92330 (0.0009) [2023-10-12 23:55:13,766][44959] Updated weights for policy 1, policy_version 92340 (0.0011) [2023-10-12 23:55:14,132][44959] Updated weights for policy 1, policy_version 92350 (0.0009) [2023-10-12 23:55:15,564][44958] Updated weights for policy 0, policy_version 91910 (0.0008) [2023-10-12 23:55:15,939][44958] Updated weights for policy 0, policy_version 91920 (0.0007) [2023-10-12 23:55:16,314][44958] Updated weights for policy 0, policy_version 91930 (0.0008) [2023-10-12 23:55:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188678144. Throughput: 0: 1640.3, 1: 1654.0. Samples: 47184626. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:16,443][43579] Avg episode reward: [(0, '282.950'), (1, '275.020')] [2023-10-12 23:55:18,291][44959] Updated weights for policy 1, policy_version 92360 (0.0009) [2023-10-12 23:55:18,663][44959] Updated weights for policy 1, policy_version 92370 (0.0009) [2023-10-12 23:55:19,034][44959] Updated weights for policy 1, policy_version 92380 (0.0007) [2023-10-12 23:55:20,539][44958] Updated weights for policy 0, policy_version 91940 (0.0008) [2023-10-12 23:55:20,922][44958] Updated weights for policy 0, policy_version 91950 (0.0008) [2023-10-12 23:55:21,303][44958] Updated weights for policy 0, policy_version 91960 (0.0009) [2023-10-12 23:55:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 188743680. Throughput: 0: 1649.2, 1: 1645.9. Samples: 47194628. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:21,443][43579] Avg episode reward: [(0, '279.920'), (1, '275.730')] [2023-10-12 23:55:23,312][44959] Updated weights for policy 1, policy_version 92390 (0.0008) [2023-10-12 23:55:23,681][44959] Updated weights for policy 1, policy_version 92400 (0.0010) [2023-10-12 23:55:24,041][44959] Updated weights for policy 1, policy_version 92410 (0.0010) [2023-10-12 23:55:25,252][44958] Updated weights for policy 0, policy_version 91970 (0.0007) [2023-10-12 23:55:25,623][44958] Updated weights for policy 0, policy_version 91980 (0.0009) [2023-10-12 23:55:25,995][44958] Updated weights for policy 0, policy_version 91990 (0.0007) [2023-10-12 23:55:26,378][44958] Updated weights for policy 0, policy_version 92000 (0.0007) [2023-10-12 23:55:26,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 188841984. Throughput: 0: 1654.0, 1: 1651.2. Samples: 47214658. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:26,443][43579] Avg episode reward: [(0, '284.370'), (1, '276.750')] [2023-10-12 23:55:28,407][44959] Updated weights for policy 1, policy_version 92420 (0.0010) [2023-10-12 23:55:28,779][44959] Updated weights for policy 1, policy_version 92430 (0.0010) [2023-10-12 23:55:29,155][44959] Updated weights for policy 1, policy_version 92440 (0.0008) [2023-10-12 23:55:30,658][44958] Updated weights for policy 0, policy_version 92010 (0.0008) [2023-10-12 23:55:31,029][44958] Updated weights for policy 0, policy_version 92020 (0.0007) [2023-10-12 23:55:31,405][44958] Updated weights for policy 0, policy_version 92030 (0.0007) [2023-10-12 23:55:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 188874752. Throughput: 0: 1649.2, 1: 1649.9. Samples: 47234048. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:31,444][43579] Avg episode reward: [(0, '281.220'), (1, '275.280')] [2023-10-12 23:55:33,082][44959] Updated weights for policy 1, policy_version 92450 (0.0009) [2023-10-12 23:55:33,452][44959] Updated weights for policy 1, policy_version 92460 (0.0009) [2023-10-12 23:55:33,813][44959] Updated weights for policy 1, policy_version 92470 (0.0010) [2023-10-12 23:55:34,183][44959] Updated weights for policy 1, policy_version 92480 (0.0008) [2023-10-12 23:55:35,344][44958] Updated weights for policy 0, policy_version 92040 (0.0007) [2023-10-12 23:55:35,716][44958] Updated weights for policy 0, policy_version 92050 (0.0007) [2023-10-12 23:55:36,088][44958] Updated weights for policy 0, policy_version 92060 (0.0007) [2023-10-12 23:55:36,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 188973056. Throughput: 0: 1656.2, 1: 1644.4. Samples: 47244196. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:36,444][43579] Avg episode reward: [(0, '287.430'), (1, '276.230')] [2023-10-12 23:55:38,353][44959] Updated weights for policy 1, policy_version 92490 (0.0008) [2023-10-12 23:55:38,724][44959] Updated weights for policy 1, policy_version 92500 (0.0010) [2023-10-12 23:55:39,096][44959] Updated weights for policy 1, policy_version 92510 (0.0009) [2023-10-12 23:55:40,176][44958] Updated weights for policy 0, policy_version 92070 (0.0007) [2023-10-12 23:55:40,546][44958] Updated weights for policy 0, policy_version 92080 (0.0008) [2023-10-12 23:55:40,922][44958] Updated weights for policy 0, policy_version 92090 (0.0007) [2023-10-12 23:55:41,442][43579] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 189038592. Throughput: 0: 1653.1, 1: 1657.2. Samples: 47264176. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:41,443][43579] Avg episode reward: [(0, '286.400'), (1, '281.640')] [2023-10-12 23:55:43,191][44959] Updated weights for policy 1, policy_version 92520 (0.0010) [2023-10-12 23:55:43,563][44959] Updated weights for policy 1, policy_version 92530 (0.0007) [2023-10-12 23:55:43,932][44959] Updated weights for policy 1, policy_version 92540 (0.0009) [2023-10-12 23:55:45,135][44958] Updated weights for policy 0, policy_version 92100 (0.0009) [2023-10-12 23:55:45,510][44958] Updated weights for policy 0, policy_version 92110 (0.0008) [2023-10-12 23:55:45,875][44958] Updated weights for policy 0, policy_version 92120 (0.0008) [2023-10-12 23:55:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 189104128. Throughput: 0: 1644.8, 1: 1657.4. Samples: 47283526. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:46,443][43579] Avg episode reward: [(0, '280.100'), (1, '281.890')] [2023-10-12 23:55:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000092544_94765056.pth... [2023-10-12 23:55:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000092128_94339072.pth... [2023-10-12 23:55:46,505][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000090592_92766208.pth [2023-10-12 23:55:46,505][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000091008_93192192.pth [2023-10-12 23:55:46,510][44518] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p0/milestones/checkpoint_000092128_94339072.pth [2023-10-12 23:55:46,511][44583] Saving a milestone ./train_atari/atari_krull_APPO/checkpoint_p1/milestones/checkpoint_000092544_94765056.pth [2023-10-12 23:55:48,024][44959] Updated weights for policy 1, policy_version 92550 (0.0010) [2023-10-12 23:55:48,388][44959] Updated weights for policy 1, policy_version 92560 (0.0008) [2023-10-12 23:55:48,764][44959] Updated weights for policy 1, policy_version 92570 (0.0007) [2023-10-12 23:55:50,000][44958] Updated weights for policy 0, policy_version 92130 (0.0008) [2023-10-12 23:55:50,374][44958] Updated weights for policy 0, policy_version 92140 (0.0010) [2023-10-12 23:55:50,742][44958] Updated weights for policy 0, policy_version 92150 (0.0007) [2023-10-12 23:55:51,112][44958] Updated weights for policy 0, policy_version 92160 (0.0007) [2023-10-12 23:55:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 189169664. Throughput: 0: 1661.3, 1: 1645.2. Samples: 47293658. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-12 23:55:51,443][43579] Avg episode reward: [(0, '278.850'), (1, '279.710')] [2023-10-12 23:55:53,250][44959] Updated weights for policy 1, policy_version 92580 (0.0008) [2023-10-12 23:55:53,617][44959] Updated weights for policy 1, policy_version 92590 (0.0009) [2023-10-12 23:55:53,990][44959] Updated weights for policy 1, policy_version 92600 (0.0008) [2023-10-12 23:55:55,348][44958] Updated weights for policy 0, policy_version 92170 (0.0008) [2023-10-12 23:55:55,723][44958] Updated weights for policy 0, policy_version 92180 (0.0008) [2023-10-12 23:55:56,096][44958] Updated weights for policy 0, policy_version 92190 (0.0008) [2023-10-12 23:55:56,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 189235200. Throughput: 0: 1655.8, 1: 1653.6. Samples: 47313912. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:55:56,443][43579] Avg episode reward: [(0, '278.600'), (1, '280.290')] [2023-10-12 23:55:58,156][44959] Updated weights for policy 1, policy_version 92610 (0.0008) [2023-10-12 23:55:58,517][44959] Updated weights for policy 1, policy_version 92620 (0.0010) [2023-10-12 23:55:58,888][44959] Updated weights for policy 1, policy_version 92630 (0.0008) [2023-10-12 23:55:59,253][44959] Updated weights for policy 1, policy_version 92640 (0.0008) [2023-10-12 23:56:00,200][44958] Updated weights for policy 0, policy_version 92200 (0.0011) [2023-10-12 23:56:00,565][44958] Updated weights for policy 0, policy_version 92210 (0.0007) [2023-10-12 23:56:00,943][44958] Updated weights for policy 0, policy_version 92220 (0.0008) [2023-10-12 23:56:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 189300736. Throughput: 0: 1647.4, 1: 1651.9. Samples: 47333094. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:01,443][43579] Avg episode reward: [(0, '275.400'), (1, '279.800')] [2023-10-12 23:56:03,406][44959] Updated weights for policy 1, policy_version 92650 (0.0010) [2023-10-12 23:56:03,779][44959] Updated weights for policy 1, policy_version 92660 (0.0008) [2023-10-12 23:56:04,138][44959] Updated weights for policy 1, policy_version 92670 (0.0008) [2023-10-12 23:56:05,006][44958] Updated weights for policy 0, policy_version 92230 (0.0008) [2023-10-12 23:56:05,389][44958] Updated weights for policy 0, policy_version 92240 (0.0009) [2023-10-12 23:56:05,762][44958] Updated weights for policy 0, policy_version 92250 (0.0010) [2023-10-12 23:56:06,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 189366272. Throughput: 0: 1655.7, 1: 1655.0. Samples: 47343610. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:06,443][43579] Avg episode reward: [(0, '272.850'), (1, '278.840')] [2023-10-12 23:56:08,186][44959] Updated weights for policy 1, policy_version 92680 (0.0009) [2023-10-12 23:56:08,561][44959] Updated weights for policy 1, policy_version 92690 (0.0008) [2023-10-12 23:56:08,931][44959] Updated weights for policy 1, policy_version 92700 (0.0008) [2023-10-12 23:56:10,002][44958] Updated weights for policy 0, policy_version 92260 (0.0009) [2023-10-12 23:56:10,391][44958] Updated weights for policy 0, policy_version 92270 (0.0007) [2023-10-12 23:56:10,774][44958] Updated weights for policy 0, policy_version 92280 (0.0009) [2023-10-12 23:56:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 189431808. Throughput: 0: 1648.9, 1: 1655.1. Samples: 47363338. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:11,443][43579] Avg episode reward: [(0, '271.750'), (1, '278.100')] [2023-10-12 23:56:12,968][44959] Updated weights for policy 1, policy_version 92710 (0.0008) [2023-10-12 23:56:13,330][44959] Updated weights for policy 1, policy_version 92720 (0.0007) [2023-10-12 23:56:13,710][44959] Updated weights for policy 1, policy_version 92730 (0.0008) [2023-10-12 23:56:14,963][44958] Updated weights for policy 0, policy_version 92290 (0.0008) [2023-10-12 23:56:15,339][44958] Updated weights for policy 0, policy_version 92300 (0.0010) [2023-10-12 23:56:15,706][44958] Updated weights for policy 0, policy_version 92310 (0.0010) [2023-10-12 23:56:16,071][44958] Updated weights for policy 0, policy_version 92320 (0.0011) [2023-10-12 23:56:16,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 189497344. Throughput: 0: 1646.7, 1: 1657.1. Samples: 47382718. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:16,443][43579] Avg episode reward: [(0, '276.310'), (1, '279.140')] [2023-10-12 23:56:17,926][44959] Updated weights for policy 1, policy_version 92740 (0.0008) [2023-10-12 23:56:18,303][44959] Updated weights for policy 1, policy_version 92750 (0.0011) [2023-10-12 23:56:18,678][44959] Updated weights for policy 1, policy_version 92760 (0.0009) [2023-10-12 23:56:20,273][44958] Updated weights for policy 0, policy_version 92330 (0.0009) [2023-10-12 23:56:20,649][44958] Updated weights for policy 0, policy_version 92340 (0.0008) [2023-10-12 23:56:21,028][44958] Updated weights for policy 0, policy_version 92350 (0.0007) [2023-10-12 23:56:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 189562880. Throughput: 0: 1653.1, 1: 1650.5. Samples: 47392858. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:21,443][43579] Avg episode reward: [(0, '278.540'), (1, '283.350')] [2023-10-12 23:56:22,805][44959] Updated weights for policy 1, policy_version 92770 (0.0007) [2023-10-12 23:56:23,167][44959] Updated weights for policy 1, policy_version 92780 (0.0009) [2023-10-12 23:56:23,538][44959] Updated weights for policy 1, policy_version 92790 (0.0009) [2023-10-12 23:56:23,904][44959] Updated weights for policy 1, policy_version 92800 (0.0009) [2023-10-12 23:56:24,908][44958] Updated weights for policy 0, policy_version 92360 (0.0007) [2023-10-12 23:56:25,292][44958] Updated weights for policy 0, policy_version 92370 (0.0009) [2023-10-12 23:56:25,657][44958] Updated weights for policy 0, policy_version 92380 (0.0009) [2023-10-12 23:56:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 189628416. Throughput: 0: 1646.6, 1: 1657.2. Samples: 47412848. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:26,443][43579] Avg episode reward: [(0, '275.160'), (1, '282.740')] [2023-10-12 23:56:28,024][44959] Updated weights for policy 1, policy_version 92810 (0.0010) [2023-10-12 23:56:28,388][44959] Updated weights for policy 1, policy_version 92820 (0.0010) [2023-10-12 23:56:28,759][44959] Updated weights for policy 1, policy_version 92830 (0.0011) [2023-10-12 23:56:29,843][44958] Updated weights for policy 0, policy_version 92390 (0.0009) [2023-10-12 23:56:30,215][44958] Updated weights for policy 0, policy_version 92400 (0.0008) [2023-10-12 23:56:30,596][44958] Updated weights for policy 0, policy_version 92410 (0.0008) [2023-10-12 23:56:31,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 189693952. Throughput: 0: 1653.8, 1: 1656.9. Samples: 47432510. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:31,443][43579] Avg episode reward: [(0, '273.750'), (1, '282.430')] [2023-10-12 23:56:32,841][44959] Updated weights for policy 1, policy_version 92840 (0.0009) [2023-10-12 23:56:33,203][44959] Updated weights for policy 1, policy_version 92850 (0.0009) [2023-10-12 23:56:33,580][44959] Updated weights for policy 1, policy_version 92860 (0.0011) [2023-10-12 23:56:34,778][44958] Updated weights for policy 0, policy_version 92420 (0.0007) [2023-10-12 23:56:35,156][44958] Updated weights for policy 0, policy_version 92430 (0.0007) [2023-10-12 23:56:35,531][44958] Updated weights for policy 0, policy_version 92440 (0.0007) [2023-10-12 23:56:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 189759488. Throughput: 0: 1653.4, 1: 1653.9. Samples: 47442486. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:36,444][43579] Avg episode reward: [(0, '275.140'), (1, '286.680')] [2023-10-12 23:56:37,825][44959] Updated weights for policy 1, policy_version 92870 (0.0008) [2023-10-12 23:56:38,193][44959] Updated weights for policy 1, policy_version 92880 (0.0008) [2023-10-12 23:56:38,559][44959] Updated weights for policy 1, policy_version 92890 (0.0011) [2023-10-12 23:56:39,705][44958] Updated weights for policy 0, policy_version 92450 (0.0007) [2023-10-12 23:56:40,074][44958] Updated weights for policy 0, policy_version 92460 (0.0009) [2023-10-12 23:56:40,459][44958] Updated weights for policy 0, policy_version 92470 (0.0007) [2023-10-12 23:56:40,829][44958] Updated weights for policy 0, policy_version 92480 (0.0007) [2023-10-12 23:56:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 189825024. Throughput: 0: 1644.7, 1: 1649.9. Samples: 47462170. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:41,443][43579] Avg episode reward: [(0, '272.460'), (1, '289.600')] [2023-10-12 23:56:42,626][44959] Updated weights for policy 1, policy_version 92900 (0.0009) [2023-10-12 23:56:42,991][44959] Updated weights for policy 1, policy_version 92910 (0.0009) [2023-10-12 23:56:43,357][44959] Updated weights for policy 1, policy_version 92920 (0.0008) [2023-10-12 23:56:44,733][44958] Updated weights for policy 0, policy_version 92490 (0.0009) [2023-10-12 23:56:45,102][44958] Updated weights for policy 0, policy_version 92500 (0.0008) [2023-10-12 23:56:45,464][44958] Updated weights for policy 0, policy_version 92510 (0.0009) [2023-10-12 23:56:46,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 189890560. Throughput: 0: 1654.2, 1: 1651.2. Samples: 47481836. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:46,443][43579] Avg episode reward: [(0, '274.440'), (1, '290.190')] [2023-10-12 23:56:47,660][44959] Updated weights for policy 1, policy_version 92930 (0.0007) [2023-10-12 23:56:48,021][44959] Updated weights for policy 1, policy_version 92940 (0.0010) [2023-10-12 23:56:48,387][44959] Updated weights for policy 1, policy_version 92950 (0.0010) [2023-10-12 23:56:48,755][44959] Updated weights for policy 1, policy_version 92960 (0.0008) [2023-10-12 23:56:49,720][44958] Updated weights for policy 0, policy_version 92520 (0.0009) [2023-10-12 23:56:50,085][44958] Updated weights for policy 0, policy_version 92530 (0.0008) [2023-10-12 23:56:50,458][44958] Updated weights for policy 0, policy_version 92540 (0.0008) [2023-10-12 23:56:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 189956096. Throughput: 0: 1652.9, 1: 1643.3. Samples: 47491938. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-12 23:56:51,443][43579] Avg episode reward: [(0, '282.410'), (1, '290.230')] [2023-10-12 23:56:52,929][44959] Updated weights for policy 1, policy_version 92970 (0.0011) [2023-10-12 23:56:53,307][44959] Updated weights for policy 1, policy_version 92980 (0.0008) [2023-10-12 23:56:53,670][44959] Updated weights for policy 1, policy_version 92990 (0.0007) [2023-10-12 23:56:54,773][44958] Updated weights for policy 0, policy_version 92550 (0.0011) [2023-10-12 23:56:55,149][44958] Updated weights for policy 0, policy_version 92560 (0.0011) [2023-10-12 23:56:55,525][44958] Updated weights for policy 0, policy_version 92570 (0.0011) [2023-10-12 23:56:56,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190021632. Throughput: 0: 1641.9, 1: 1653.9. Samples: 47511646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:56:56,444][43579] Avg episode reward: [(0, '281.250'), (1, '286.670')] [2023-10-12 23:56:57,819][44959] Updated weights for policy 1, policy_version 93000 (0.0007) [2023-10-12 23:56:58,206][44959] Updated weights for policy 1, policy_version 93010 (0.0008) [2023-10-12 23:56:58,577][44959] Updated weights for policy 1, policy_version 93020 (0.0009) [2023-10-12 23:56:59,602][44958] Updated weights for policy 0, policy_version 92580 (0.0010) [2023-10-12 23:56:59,977][44958] Updated weights for policy 0, policy_version 92590 (0.0009) [2023-10-12 23:57:00,344][44958] Updated weights for policy 0, policy_version 92600 (0.0009) [2023-10-12 23:57:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190087168. Throughput: 0: 1650.1, 1: 1650.1. Samples: 47531228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:01,444][43579] Avg episode reward: [(0, '285.690'), (1, '284.220')] [2023-10-12 23:57:02,706][44959] Updated weights for policy 1, policy_version 93030 (0.0009) [2023-10-12 23:57:03,065][44959] Updated weights for policy 1, policy_version 93040 (0.0008) [2023-10-12 23:57:03,440][44959] Updated weights for policy 1, policy_version 93050 (0.0009) [2023-10-12 23:57:04,508][44958] Updated weights for policy 0, policy_version 92610 (0.0009) [2023-10-12 23:57:04,878][44958] Updated weights for policy 0, policy_version 92620 (0.0008) [2023-10-12 23:57:05,249][44958] Updated weights for policy 0, policy_version 92630 (0.0011) [2023-10-12 23:57:05,619][44958] Updated weights for policy 0, policy_version 92640 (0.0011) [2023-10-12 23:57:06,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190152704. Throughput: 0: 1653.7, 1: 1652.3. Samples: 47541628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:06,444][43579] Avg episode reward: [(0, '285.490'), (1, '281.430')] [2023-10-12 23:57:07,471][44959] Updated weights for policy 1, policy_version 93060 (0.0008) [2023-10-12 23:57:07,829][44959] Updated weights for policy 1, policy_version 93070 (0.0008) [2023-10-12 23:57:08,193][44959] Updated weights for policy 1, policy_version 93080 (0.0008) [2023-10-12 23:57:09,620][44958] Updated weights for policy 0, policy_version 92650 (0.0010) [2023-10-12 23:57:09,987][44958] Updated weights for policy 0, policy_version 92660 (0.0009) [2023-10-12 23:57:10,368][44958] Updated weights for policy 0, policy_version 92670 (0.0009) [2023-10-12 23:57:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 190218240. Throughput: 0: 1647.4, 1: 1652.9. Samples: 47561362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:11,444][43579] Avg episode reward: [(0, '287.590'), (1, '274.400')] [2023-10-12 23:57:12,418][44959] Updated weights for policy 1, policy_version 93090 (0.0008) [2023-10-12 23:57:12,789][44959] Updated weights for policy 1, policy_version 93100 (0.0010) [2023-10-12 23:57:13,153][44959] Updated weights for policy 1, policy_version 93110 (0.0010) [2023-10-12 23:57:13,519][44959] Updated weights for policy 1, policy_version 93120 (0.0010) [2023-10-12 23:57:14,630][44958] Updated weights for policy 0, policy_version 92680 (0.0009) [2023-10-12 23:57:14,994][44958] Updated weights for policy 0, policy_version 92690 (0.0008) [2023-10-12 23:57:15,364][44958] Updated weights for policy 0, policy_version 92700 (0.0007) [2023-10-12 23:57:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190283776. Throughput: 0: 1651.9, 1: 1648.3. Samples: 47581018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:16,443][43579] Avg episode reward: [(0, '285.140'), (1, '270.130')] [2023-10-12 23:57:17,756][44959] Updated weights for policy 1, policy_version 93130 (0.0008) [2023-10-12 23:57:18,118][44959] Updated weights for policy 1, policy_version 93140 (0.0008) [2023-10-12 23:57:18,491][44959] Updated weights for policy 1, policy_version 93150 (0.0008) [2023-10-12 23:57:19,504][44958] Updated weights for policy 0, policy_version 92710 (0.0009) [2023-10-12 23:57:19,878][44958] Updated weights for policy 0, policy_version 92720 (0.0007) [2023-10-12 23:57:20,250][44958] Updated weights for policy 0, policy_version 92730 (0.0008) [2023-10-12 23:57:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 190349312. Throughput: 0: 1655.6, 1: 1653.2. Samples: 47591382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:21,444][43579] Avg episode reward: [(0, '283.990'), (1, '268.760')] [2023-10-12 23:57:22,572][44959] Updated weights for policy 1, policy_version 93160 (0.0007) [2023-10-12 23:57:22,932][44959] Updated weights for policy 1, policy_version 93170 (0.0007) [2023-10-12 23:57:23,298][44959] Updated weights for policy 1, policy_version 93180 (0.0008) [2023-10-12 23:57:24,403][44958] Updated weights for policy 0, policy_version 92740 (0.0008) [2023-10-12 23:57:24,767][44958] Updated weights for policy 0, policy_version 92750 (0.0010) [2023-10-12 23:57:25,141][44958] Updated weights for policy 0, policy_version 92760 (0.0010) [2023-10-12 23:57:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190414848. Throughput: 0: 1642.4, 1: 1660.4. Samples: 47610792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:26,443][43579] Avg episode reward: [(0, '282.390'), (1, '273.300')] [2023-10-12 23:57:27,527][44959] Updated weights for policy 1, policy_version 93190 (0.0008) [2023-10-12 23:57:27,893][44959] Updated weights for policy 1, policy_version 93200 (0.0009) [2023-10-12 23:57:28,271][44959] Updated weights for policy 1, policy_version 93210 (0.0008) [2023-10-12 23:57:29,298][44958] Updated weights for policy 0, policy_version 92770 (0.0011) [2023-10-12 23:57:29,665][44958] Updated weights for policy 0, policy_version 92780 (0.0011) [2023-10-12 23:57:30,046][44958] Updated weights for policy 0, policy_version 92790 (0.0009) [2023-10-12 23:57:30,419][44958] Updated weights for policy 0, policy_version 92800 (0.0009) [2023-10-12 23:57:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190480384. Throughput: 0: 1643.7, 1: 1662.6. Samples: 47630622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:31,444][43579] Avg episode reward: [(0, '284.260'), (1, '269.140')] [2023-10-12 23:57:32,094][44959] Updated weights for policy 1, policy_version 93220 (0.0009) [2023-10-12 23:57:32,466][44959] Updated weights for policy 1, policy_version 93230 (0.0010) [2023-10-12 23:57:32,836][44959] Updated weights for policy 1, policy_version 93240 (0.0008) [2023-10-12 23:57:34,576][44958] Updated weights for policy 0, policy_version 92810 (0.0009) [2023-10-12 23:57:34,953][44958] Updated weights for policy 0, policy_version 92820 (0.0011) [2023-10-12 23:57:35,335][44958] Updated weights for policy 0, policy_version 92830 (0.0011) [2023-10-12 23:57:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190545920. Throughput: 0: 1645.0, 1: 1665.6. Samples: 47640914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:36,443][43579] Avg episode reward: [(0, '278.570'), (1, '269.270')] [2023-10-12 23:57:37,069][44959] Updated weights for policy 1, policy_version 93250 (0.0009) [2023-10-12 23:57:37,444][44959] Updated weights for policy 1, policy_version 93260 (0.0009) [2023-10-12 23:57:37,819][44959] Updated weights for policy 1, policy_version 93270 (0.0007) [2023-10-12 23:57:38,181][44959] Updated weights for policy 1, policy_version 93280 (0.0008) [2023-10-12 23:57:39,718][44958] Updated weights for policy 0, policy_version 92840 (0.0009) [2023-10-12 23:57:40,081][44958] Updated weights for policy 0, policy_version 92850 (0.0010) [2023-10-12 23:57:40,445][44958] Updated weights for policy 0, policy_version 92860 (0.0007) [2023-10-12 23:57:41,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190611456. Throughput: 0: 1636.0, 1: 1667.6. Samples: 47660308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:41,444][43579] Avg episode reward: [(0, '280.980'), (1, '274.130')] [2023-10-12 23:57:42,222][44959] Updated weights for policy 1, policy_version 93290 (0.0009) [2023-10-12 23:57:42,591][44959] Updated weights for policy 1, policy_version 93300 (0.0007) [2023-10-12 23:57:42,955][44959] Updated weights for policy 1, policy_version 93310 (0.0008) [2023-10-12 23:57:44,692][44958] Updated weights for policy 0, policy_version 92870 (0.0007) [2023-10-12 23:57:45,060][44958] Updated weights for policy 0, policy_version 92880 (0.0009) [2023-10-12 23:57:45,441][44958] Updated weights for policy 0, policy_version 92890 (0.0008) [2023-10-12 23:57:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190676992. Throughput: 0: 1633.2, 1: 1673.3. Samples: 47680022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:46,443][43579] Avg episode reward: [(0, '283.950'), (1, '276.140')] [2023-10-12 23:57:46,452][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000093312_95551488.pth... [2023-10-12 23:57:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000092896_95125504.pth... [2023-10-12 23:57:46,488][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000091776_93978624.pth [2023-10-12 23:57:46,491][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000091360_93552640.pth [2023-10-12 23:57:47,087][44959] Updated weights for policy 1, policy_version 93320 (0.0010) [2023-10-12 23:57:47,475][44959] Updated weights for policy 1, policy_version 93330 (0.0011) [2023-10-12 23:57:47,836][44959] Updated weights for policy 1, policy_version 93340 (0.0008) [2023-10-12 23:57:49,711][44958] Updated weights for policy 0, policy_version 92900 (0.0009) [2023-10-12 23:57:50,083][44958] Updated weights for policy 0, policy_version 92910 (0.0011) [2023-10-12 23:57:50,453][44958] Updated weights for policy 0, policy_version 92920 (0.0010) [2023-10-12 23:57:51,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190742528. Throughput: 0: 1630.1, 1: 1663.7. Samples: 47689850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:57:51,443][43579] Avg episode reward: [(0, '285.730'), (1, '275.930')] [2023-10-12 23:57:51,936][44959] Updated weights for policy 1, policy_version 93350 (0.0010) [2023-10-12 23:57:52,312][44959] Updated weights for policy 1, policy_version 93360 (0.0008) [2023-10-12 23:57:52,677][44959] Updated weights for policy 1, policy_version 93370 (0.0009) [2023-10-12 23:57:54,980][44958] Updated weights for policy 0, policy_version 92930 (0.0009) [2023-10-12 23:57:55,358][44958] Updated weights for policy 0, policy_version 92940 (0.0009) [2023-10-12 23:57:55,722][44958] Updated weights for policy 0, policy_version 92950 (0.0008) [2023-10-12 23:57:56,105][44958] Updated weights for policy 0, policy_version 92960 (0.0009) [2023-10-12 23:57:56,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 190808064. Throughput: 0: 1629.7, 1: 1660.5. Samples: 47709420. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:57:56,443][43579] Avg episode reward: [(0, '288.380'), (1, '276.800')] [2023-10-12 23:57:56,815][44959] Updated weights for policy 1, policy_version 93380 (0.0008) [2023-10-12 23:57:57,184][44959] Updated weights for policy 1, policy_version 93390 (0.0009) [2023-10-12 23:57:57,549][44959] Updated weights for policy 1, policy_version 93400 (0.0010) [2023-10-12 23:58:00,209][44958] Updated weights for policy 0, policy_version 92970 (0.0008) [2023-10-12 23:58:00,584][44958] Updated weights for policy 0, policy_version 92980 (0.0007) [2023-10-12 23:58:00,964][44958] Updated weights for policy 0, policy_version 92990 (0.0009) [2023-10-12 23:58:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190873600. Throughput: 0: 1613.5, 1: 1663.8. Samples: 47728496. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:01,443][43579] Avg episode reward: [(0, '285.730'), (1, '275.650')] [2023-10-12 23:58:01,878][44959] Updated weights for policy 1, policy_version 93410 (0.0008) [2023-10-12 23:58:02,244][44959] Updated weights for policy 1, policy_version 93420 (0.0007) [2023-10-12 23:58:02,614][44959] Updated weights for policy 1, policy_version 93430 (0.0008) [2023-10-12 23:58:02,978][44959] Updated weights for policy 1, policy_version 93440 (0.0011) [2023-10-12 23:58:05,289][44958] Updated weights for policy 0, policy_version 93000 (0.0009) [2023-10-12 23:58:05,644][44958] Updated weights for policy 0, policy_version 93010 (0.0008) [2023-10-12 23:58:06,008][44958] Updated weights for policy 0, policy_version 93020 (0.0008) [2023-10-12 23:58:06,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 190939136. Throughput: 0: 1611.9, 1: 1661.6. Samples: 47738690. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:06,444][43579] Avg episode reward: [(0, '284.680'), (1, '274.360')] [2023-10-12 23:58:06,937][44959] Updated weights for policy 1, policy_version 93450 (0.0009) [2023-10-12 23:58:07,307][44959] Updated weights for policy 1, policy_version 93460 (0.0010) [2023-10-12 23:58:07,687][44959] Updated weights for policy 1, policy_version 93470 (0.0011) [2023-10-12 23:58:10,164][44958] Updated weights for policy 0, policy_version 93030 (0.0008) [2023-10-12 23:58:10,538][44958] Updated weights for policy 0, policy_version 93040 (0.0007) [2023-10-12 23:58:10,910][44958] Updated weights for policy 0, policy_version 93050 (0.0010) [2023-10-12 23:58:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191004672. Throughput: 0: 1628.4, 1: 1655.6. Samples: 47758574. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:11,443][43579] Avg episode reward: [(0, '283.900'), (1, '274.780')] [2023-10-12 23:58:12,055][44959] Updated weights for policy 1, policy_version 93480 (0.0008) [2023-10-12 23:58:12,429][44959] Updated weights for policy 1, policy_version 93490 (0.0008) [2023-10-12 23:58:12,791][44959] Updated weights for policy 1, policy_version 93500 (0.0009) [2023-10-12 23:58:15,111][44958] Updated weights for policy 0, policy_version 93060 (0.0009) [2023-10-12 23:58:15,485][44958] Updated weights for policy 0, policy_version 93070 (0.0007) [2023-10-12 23:58:15,857][44958] Updated weights for policy 0, policy_version 93080 (0.0009) [2023-10-12 23:58:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 191070208. Throughput: 0: 1618.0, 1: 1651.4. Samples: 47777746. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:16,444][43579] Avg episode reward: [(0, '279.250'), (1, '273.200')] [2023-10-12 23:58:16,952][44959] Updated weights for policy 1, policy_version 93510 (0.0011) [2023-10-12 23:58:17,321][44959] Updated weights for policy 1, policy_version 93520 (0.0009) [2023-10-12 23:58:17,706][44959] Updated weights for policy 1, policy_version 93530 (0.0009) [2023-10-12 23:58:19,899][44958] Updated weights for policy 0, policy_version 93090 (0.0007) [2023-10-12 23:58:20,264][44958] Updated weights for policy 0, policy_version 93100 (0.0008) [2023-10-12 23:58:20,646][44958] Updated weights for policy 0, policy_version 93110 (0.0008) [2023-10-12 23:58:21,016][44958] Updated weights for policy 0, policy_version 93120 (0.0007) [2023-10-12 23:58:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191135744. Throughput: 0: 1613.2, 1: 1650.8. Samples: 47787798. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:21,444][43579] Avg episode reward: [(0, '280.450'), (1, '273.020')] [2023-10-12 23:58:21,816][44959] Updated weights for policy 1, policy_version 93540 (0.0008) [2023-10-12 23:58:22,193][44959] Updated weights for policy 1, policy_version 93550 (0.0009) [2023-10-12 23:58:22,559][44959] Updated weights for policy 1, policy_version 93560 (0.0009) [2023-10-12 23:58:25,254][44958] Updated weights for policy 0, policy_version 93130 (0.0009) [2023-10-12 23:58:25,627][44958] Updated weights for policy 0, policy_version 93140 (0.0009) [2023-10-12 23:58:26,000][44958] Updated weights for policy 0, policy_version 93150 (0.0007) [2023-10-12 23:58:26,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191201280. Throughput: 0: 1627.7, 1: 1644.5. Samples: 47807560. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:26,443][43579] Avg episode reward: [(0, '278.540'), (1, '275.550')] [2023-10-12 23:58:26,889][44959] Updated weights for policy 1, policy_version 93570 (0.0010) [2023-10-12 23:58:27,261][44959] Updated weights for policy 1, policy_version 93580 (0.0008) [2023-10-12 23:58:27,624][44959] Updated weights for policy 1, policy_version 93590 (0.0008) [2023-10-12 23:58:27,988][44959] Updated weights for policy 1, policy_version 93600 (0.0009) [2023-10-12 23:58:30,151][44958] Updated weights for policy 0, policy_version 93160 (0.0007) [2023-10-12 23:58:30,531][44958] Updated weights for policy 0, policy_version 93170 (0.0010) [2023-10-12 23:58:30,894][44958] Updated weights for policy 0, policy_version 93180 (0.0011) [2023-10-12 23:58:31,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191266816. Throughput: 0: 1621.2, 1: 1638.6. Samples: 47826712. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:31,444][43579] Avg episode reward: [(0, '277.610'), (1, '280.720')] [2023-10-12 23:58:32,103][44959] Updated weights for policy 1, policy_version 93610 (0.0009) [2023-10-12 23:58:32,481][44959] Updated weights for policy 1, policy_version 93620 (0.0008) [2023-10-12 23:58:32,854][44959] Updated weights for policy 1, policy_version 93630 (0.0008) [2023-10-12 23:58:34,950][44958] Updated weights for policy 0, policy_version 93190 (0.0009) [2023-10-12 23:58:35,325][44958] Updated weights for policy 0, policy_version 93200 (0.0009) [2023-10-12 23:58:35,704][44958] Updated weights for policy 0, policy_version 93210 (0.0009) [2023-10-12 23:58:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 191332352. Throughput: 0: 1624.0, 1: 1642.5. Samples: 47836846. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:36,444][43579] Avg episode reward: [(0, '281.690'), (1, '277.280')] [2023-10-12 23:58:37,122][44959] Updated weights for policy 1, policy_version 93640 (0.0010) [2023-10-12 23:58:37,487][44959] Updated weights for policy 1, policy_version 93650 (0.0008) [2023-10-12 23:58:37,860][44959] Updated weights for policy 1, policy_version 93660 (0.0008) [2023-10-12 23:58:40,075][44958] Updated weights for policy 0, policy_version 93220 (0.0010) [2023-10-12 23:58:40,445][44958] Updated weights for policy 0, policy_version 93230 (0.0008) [2023-10-12 23:58:40,818][44958] Updated weights for policy 0, policy_version 93240 (0.0008) [2023-10-12 23:58:41,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191397888. Throughput: 0: 1631.3, 1: 1648.3. Samples: 47857000. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:41,444][43579] Avg episode reward: [(0, '286.690'), (1, '278.750')] [2023-10-12 23:58:41,778][44959] Updated weights for policy 1, policy_version 93670 (0.0007) [2023-10-12 23:58:42,147][44959] Updated weights for policy 1, policy_version 93680 (0.0007) [2023-10-12 23:58:42,512][44959] Updated weights for policy 1, policy_version 93690 (0.0008) [2023-10-12 23:58:45,045][44958] Updated weights for policy 0, policy_version 93250 (0.0009) [2023-10-12 23:58:45,424][44958] Updated weights for policy 0, policy_version 93260 (0.0007) [2023-10-12 23:58:45,791][44958] Updated weights for policy 0, policy_version 93270 (0.0008) [2023-10-12 23:58:46,172][44958] Updated weights for policy 0, policy_version 93280 (0.0007) [2023-10-12 23:58:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191463424. Throughput: 0: 1638.3, 1: 1643.2. Samples: 47876166. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:46,444][43579] Avg episode reward: [(0, '289.020'), (1, '279.440')] [2023-10-12 23:58:46,774][44959] Updated weights for policy 1, policy_version 93700 (0.0008) [2023-10-12 23:58:47,154][44959] Updated weights for policy 1, policy_version 93710 (0.0008) [2023-10-12 23:58:47,526][44959] Updated weights for policy 1, policy_version 93720 (0.0008) [2023-10-12 23:58:50,450][44958] Updated weights for policy 0, policy_version 93290 (0.0010) [2023-10-12 23:58:50,831][44958] Updated weights for policy 0, policy_version 93300 (0.0008) [2023-10-12 23:58:51,214][44958] Updated weights for policy 0, policy_version 93310 (0.0007) [2023-10-12 23:58:51,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191528960. Throughput: 0: 1630.0, 1: 1641.3. Samples: 47885900. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-12 23:58:51,443][43579] Avg episode reward: [(0, '280.640'), (1, '274.030')] [2023-10-12 23:58:51,631][44959] Updated weights for policy 1, policy_version 93730 (0.0008) [2023-10-12 23:58:51,992][44959] Updated weights for policy 1, policy_version 93740 (0.0009) [2023-10-12 23:58:52,370][44959] Updated weights for policy 1, policy_version 93750 (0.0007) [2023-10-12 23:58:52,730][44959] Updated weights for policy 1, policy_version 93760 (0.0008) [2023-10-12 23:58:55,288][44958] Updated weights for policy 0, policy_version 93320 (0.0009) [2023-10-12 23:58:55,656][44958] Updated weights for policy 0, policy_version 93330 (0.0009) [2023-10-12 23:58:56,025][44958] Updated weights for policy 0, policy_version 93340 (0.0008) [2023-10-12 23:58:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191594496. Throughput: 0: 1631.6, 1: 1650.1. Samples: 47906252. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:58:56,444][43579] Avg episode reward: [(0, '286.300'), (1, '274.140')] [2023-10-12 23:58:56,993][44959] Updated weights for policy 1, policy_version 93770 (0.0009) [2023-10-12 23:58:57,370][44959] Updated weights for policy 1, policy_version 93780 (0.0009) [2023-10-12 23:58:57,739][44959] Updated weights for policy 1, policy_version 93790 (0.0007) [2023-10-12 23:59:00,372][44958] Updated weights for policy 0, policy_version 93350 (0.0009) [2023-10-12 23:59:00,743][44958] Updated weights for policy 0, policy_version 93360 (0.0009) [2023-10-12 23:59:01,124][44958] Updated weights for policy 0, policy_version 93370 (0.0009) [2023-10-12 23:59:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191660032. Throughput: 0: 1634.2, 1: 1651.2. Samples: 47925586. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:01,444][43579] Avg episode reward: [(0, '282.710'), (1, '271.490')] [2023-10-12 23:59:01,707][44959] Updated weights for policy 1, policy_version 93800 (0.0009) [2023-10-12 23:59:02,073][44959] Updated weights for policy 1, policy_version 93810 (0.0009) [2023-10-12 23:59:02,455][44959] Updated weights for policy 1, policy_version 93820 (0.0009) [2023-10-12 23:59:05,342][44958] Updated weights for policy 0, policy_version 93380 (0.0010) [2023-10-12 23:59:05,710][44958] Updated weights for policy 0, policy_version 93390 (0.0010) [2023-10-12 23:59:06,088][44958] Updated weights for policy 0, policy_version 93400 (0.0008) [2023-10-12 23:59:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191725568. Throughput: 0: 1632.2, 1: 1650.6. Samples: 47935524. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:06,443][43579] Avg episode reward: [(0, '280.190'), (1, '270.740')] [2023-10-12 23:59:06,519][44959] Updated weights for policy 1, policy_version 93830 (0.0010) [2023-10-12 23:59:06,879][44959] Updated weights for policy 1, policy_version 93840 (0.0009) [2023-10-12 23:59:07,241][44959] Updated weights for policy 1, policy_version 93850 (0.0008) [2023-10-12 23:59:10,295][44958] Updated weights for policy 0, policy_version 93410 (0.0009) [2023-10-12 23:59:10,708][44958] Updated weights for policy 0, policy_version 93420 (0.0009) [2023-10-12 23:59:11,093][44958] Updated weights for policy 0, policy_version 93430 (0.0008) [2023-10-12 23:59:11,443][43579] Fps is (10 sec: 9830.2, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 191758336. Throughput: 0: 1642.5, 1: 1654.2. Samples: 47955914. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:11,444][43579] Avg episode reward: [(0, '279.150'), (1, '270.850')] [2023-10-12 23:59:11,468][44958] Updated weights for policy 0, policy_version 93440 (0.0009) [2023-10-12 23:59:11,571][44959] Updated weights for policy 1, policy_version 93860 (0.0009) [2023-10-12 23:59:11,947][44959] Updated weights for policy 1, policy_version 93870 (0.0008) [2023-10-12 23:59:12,318][44959] Updated weights for policy 1, policy_version 93880 (0.0008) [2023-10-12 23:59:15,614][44958] Updated weights for policy 0, policy_version 93450 (0.0008) [2023-10-12 23:59:15,978][44958] Updated weights for policy 0, policy_version 93460 (0.0007) [2023-10-12 23:59:16,347][44958] Updated weights for policy 0, policy_version 93470 (0.0007) [2023-10-12 23:59:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 191856640. Throughput: 0: 1641.4, 1: 1653.6. Samples: 47974990. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:16,443][43579] Avg episode reward: [(0, '273.290'), (1, '272.600')] [2023-10-12 23:59:16,457][44959] Updated weights for policy 1, policy_version 93890 (0.0007) [2023-10-12 23:59:16,870][44959] Updated weights for policy 1, policy_version 93900 (0.0007) [2023-10-12 23:59:17,231][44959] Updated weights for policy 1, policy_version 93910 (0.0009) [2023-10-12 23:59:17,603][44959] Updated weights for policy 1, policy_version 93920 (0.0008) [2023-10-12 23:59:20,553][44958] Updated weights for policy 0, policy_version 93480 (0.0008) [2023-10-12 23:59:20,933][44958] Updated weights for policy 0, policy_version 93490 (0.0007) [2023-10-12 23:59:21,301][44958] Updated weights for policy 0, policy_version 93500 (0.0008) [2023-10-12 23:59:21,443][43579] Fps is (10 sec: 13107.6, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 191889408. Throughput: 0: 1627.6, 1: 1651.8. Samples: 47984418. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:21,443][43579] Avg episode reward: [(0, '278.160'), (1, '276.620')] [2023-10-12 23:59:21,812][44959] Updated weights for policy 1, policy_version 93930 (0.0008) [2023-10-12 23:59:22,187][44959] Updated weights for policy 1, policy_version 93940 (0.0007) [2023-10-12 23:59:22,559][44959] Updated weights for policy 1, policy_version 93950 (0.0009) [2023-10-12 23:59:25,391][44958] Updated weights for policy 0, policy_version 93510 (0.0009) [2023-10-12 23:59:25,764][44958] Updated weights for policy 0, policy_version 93520 (0.0007) [2023-10-12 23:59:26,134][44958] Updated weights for policy 0, policy_version 93530 (0.0007) [2023-10-12 23:59:26,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 191987712. Throughput: 0: 1637.2, 1: 1646.3. Samples: 48004760. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:26,444][43579] Avg episode reward: [(0, '273.250'), (1, '270.570')] [2023-10-12 23:59:26,736][44959] Updated weights for policy 1, policy_version 93960 (0.0010) [2023-10-12 23:59:27,111][44959] Updated weights for policy 1, policy_version 93970 (0.0008) [2023-10-12 23:59:27,486][44959] Updated weights for policy 1, policy_version 93980 (0.0008) [2023-10-12 23:59:30,323][44958] Updated weights for policy 0, policy_version 93540 (0.0008) [2023-10-12 23:59:30,687][44958] Updated weights for policy 0, policy_version 93550 (0.0007) [2023-10-12 23:59:31,062][44958] Updated weights for policy 0, policy_version 93560 (0.0008) [2023-10-12 23:59:31,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 192053248. Throughput: 0: 1636.7, 1: 1646.5. Samples: 48023910. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:31,443][43579] Avg episode reward: [(0, '276.720'), (1, '278.580')] [2023-10-12 23:59:31,683][44959] Updated weights for policy 1, policy_version 93990 (0.0008) [2023-10-12 23:59:32,052][44959] Updated weights for policy 1, policy_version 94000 (0.0008) [2023-10-12 23:59:32,410][44959] Updated weights for policy 1, policy_version 94010 (0.0009) [2023-10-12 23:59:35,345][44958] Updated weights for policy 0, policy_version 93570 (0.0009) [2023-10-12 23:59:35,721][44958] Updated weights for policy 0, policy_version 93580 (0.0009) [2023-10-12 23:59:36,091][44958] Updated weights for policy 0, policy_version 93590 (0.0010) [2023-10-12 23:59:36,443][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 192086016. Throughput: 0: 1637.2, 1: 1645.2. Samples: 48033604. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:36,444][43579] Avg episode reward: [(0, '279.950'), (1, '277.610')] [2023-10-12 23:59:36,460][44958] Updated weights for policy 0, policy_version 93600 (0.0009) [2023-10-12 23:59:36,644][44959] Updated weights for policy 1, policy_version 94020 (0.0008) [2023-10-12 23:59:37,012][44959] Updated weights for policy 1, policy_version 94030 (0.0007) [2023-10-12 23:59:37,376][44959] Updated weights for policy 1, policy_version 94040 (0.0007) [2023-10-12 23:59:40,650][44958] Updated weights for policy 0, policy_version 93610 (0.0008) [2023-10-12 23:59:41,014][44958] Updated weights for policy 0, policy_version 93620 (0.0007) [2023-10-12 23:59:41,355][44959] Updated weights for policy 1, policy_version 94050 (0.0007) [2023-10-12 23:59:41,386][44958] Updated weights for policy 0, policy_version 93630 (0.0009) [2023-10-12 23:59:41,442][43579] Fps is (10 sec: 9830.5, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 192151552. Throughput: 0: 1641.1, 1: 1639.7. Samples: 48053888. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:41,443][43579] Avg episode reward: [(0, '281.050'), (1, '275.280')] [2023-10-12 23:59:41,733][44959] Updated weights for policy 1, policy_version 94060 (0.0011) [2023-10-12 23:59:42,100][44959] Updated weights for policy 1, policy_version 94070 (0.0009) [2023-10-12 23:59:42,461][44959] Updated weights for policy 1, policy_version 94080 (0.0008) [2023-10-12 23:59:45,344][44958] Updated weights for policy 0, policy_version 93640 (0.0008) [2023-10-12 23:59:45,718][44958] Updated weights for policy 0, policy_version 93650 (0.0009) [2023-10-12 23:59:46,080][44958] Updated weights for policy 0, policy_version 93660 (0.0007) [2023-10-12 23:59:46,442][43579] Fps is (10 sec: 16384.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 192249856. Throughput: 0: 1638.9, 1: 1645.6. Samples: 48073386. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:46,443][43579] Avg episode reward: [(0, '290.330'), (1, '277.920')] [2023-10-12 23:59:46,450][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000093664_95911936.pth... [2023-10-12 23:59:46,461][44959] Updated weights for policy 1, policy_version 94090 (0.0009) [2023-10-12 23:59:46,482][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000092128_94339072.pth [2023-10-12 23:59:46,822][44959] Updated weights for policy 1, policy_version 94100 (0.0010) [2023-10-12 23:59:47,204][44959] Updated weights for policy 1, policy_version 94110 (0.0011) [2023-10-12 23:59:47,269][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000094112_96370688.pth... [2023-10-12 23:59:47,309][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000092544_94765056.pth [2023-10-12 23:59:50,246][44958] Updated weights for policy 0, policy_version 93670 (0.0007) [2023-10-12 23:59:50,625][44958] Updated weights for policy 0, policy_version 93680 (0.0008) [2023-10-12 23:59:50,999][44958] Updated weights for policy 0, policy_version 93690 (0.0008) [2023-10-12 23:59:51,392][44959] Updated weights for policy 1, policy_version 94120 (0.0008) [2023-10-12 23:59:51,443][43579] Fps is (10 sec: 16383.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 192315392. Throughput: 0: 1639.7, 1: 1642.8. Samples: 48083236. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-12 23:59:51,444][43579] Avg episode reward: [(0, '291.850'), (1, '278.410')] [2023-10-12 23:59:51,445][44518] Saving new best policy, reward=291.850! [2023-10-12 23:59:51,764][44959] Updated weights for policy 1, policy_version 94130 (0.0008) [2023-10-12 23:59:52,133][44959] Updated weights for policy 1, policy_version 94140 (0.0008) [2023-10-12 23:59:55,175][44958] Updated weights for policy 0, policy_version 93700 (0.0008) [2023-10-12 23:59:55,565][44958] Updated weights for policy 0, policy_version 93710 (0.0008) [2023-10-12 23:59:55,937][44958] Updated weights for policy 0, policy_version 93720 (0.0008) [2023-10-12 23:59:56,386][44959] Updated weights for policy 1, policy_version 94150 (0.0008) [2023-10-12 23:59:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 192380928. Throughput: 0: 1636.0, 1: 1640.2. Samples: 48103342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-12 23:59:56,443][43579] Avg episode reward: [(0, '297.260'), (1, '284.490')] [2023-10-12 23:59:56,444][44518] Saving new best policy, reward=297.260! [2023-10-12 23:59:56,756][44959] Updated weights for policy 1, policy_version 94160 (0.0008) [2023-10-12 23:59:57,129][44959] Updated weights for policy 1, policy_version 94170 (0.0009) [2023-10-13 00:00:00,024][44958] Updated weights for policy 0, policy_version 93730 (0.0008) [2023-10-13 00:00:00,397][44958] Updated weights for policy 0, policy_version 93740 (0.0008) [2023-10-13 00:00:00,768][44958] Updated weights for policy 0, policy_version 93750 (0.0008) [2023-10-13 00:00:01,140][44958] Updated weights for policy 0, policy_version 93760 (0.0009) [2023-10-13 00:00:01,370][44959] Updated weights for policy 1, policy_version 94180 (0.0010) [2023-10-13 00:00:01,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 192446464. Throughput: 0: 1639.1, 1: 1643.7. Samples: 48122718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:01,443][43579] Avg episode reward: [(0, '295.890'), (1, '284.730')] [2023-10-13 00:00:01,739][44959] Updated weights for policy 1, policy_version 94190 (0.0007) [2023-10-13 00:00:02,107][44959] Updated weights for policy 1, policy_version 94200 (0.0009) [2023-10-13 00:00:05,165][44958] Updated weights for policy 0, policy_version 93770 (0.0008) [2023-10-13 00:00:05,539][44958] Updated weights for policy 0, policy_version 93780 (0.0008) [2023-10-13 00:00:05,915][44958] Updated weights for policy 0, policy_version 93790 (0.0008) [2023-10-13 00:00:06,361][44959] Updated weights for policy 1, policy_version 94210 (0.0008) [2023-10-13 00:00:06,443][43579] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 192512000. Throughput: 0: 1651.4, 1: 1653.0. Samples: 48133114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:06,444][43579] Avg episode reward: [(0, '291.380'), (1, '289.950')] [2023-10-13 00:00:06,763][44959] Updated weights for policy 1, policy_version 94220 (0.0008) [2023-10-13 00:00:07,124][44959] Updated weights for policy 1, policy_version 94230 (0.0007) [2023-10-13 00:00:07,492][44959] Updated weights for policy 1, policy_version 94240 (0.0008) [2023-10-13 00:00:10,019][44958] Updated weights for policy 0, policy_version 93800 (0.0007) [2023-10-13 00:00:10,401][44958] Updated weights for policy 0, policy_version 93810 (0.0008) [2023-10-13 00:00:10,769][44958] Updated weights for policy 0, policy_version 93820 (0.0009) [2023-10-13 00:00:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 192577536. Throughput: 0: 1643.9, 1: 1650.7. Samples: 48153016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:11,444][43579] Avg episode reward: [(0, '289.290'), (1, '290.640')] [2023-10-13 00:00:11,516][44959] Updated weights for policy 1, policy_version 94250 (0.0009) [2023-10-13 00:00:11,885][44959] Updated weights for policy 1, policy_version 94260 (0.0008) [2023-10-13 00:00:12,256][44959] Updated weights for policy 1, policy_version 94270 (0.0007) [2023-10-13 00:00:14,942][44958] Updated weights for policy 0, policy_version 93830 (0.0007) [2023-10-13 00:00:15,319][44958] Updated weights for policy 0, policy_version 93840 (0.0010) [2023-10-13 00:00:15,687][44958] Updated weights for policy 0, policy_version 93850 (0.0008) [2023-10-13 00:00:16,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 192643072. Throughput: 0: 1649.9, 1: 1652.8. Samples: 48172530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:16,443][43579] Avg episode reward: [(0, '283.830'), (1, '290.220')] [2023-10-13 00:00:16,540][44959] Updated weights for policy 1, policy_version 94280 (0.0009) [2023-10-13 00:00:16,905][44959] Updated weights for policy 1, policy_version 94290 (0.0010) [2023-10-13 00:00:17,277][44959] Updated weights for policy 1, policy_version 94300 (0.0010) [2023-10-13 00:00:19,650][44958] Updated weights for policy 0, policy_version 93860 (0.0008) [2023-10-13 00:00:20,023][44958] Updated weights for policy 0, policy_version 93870 (0.0008) [2023-10-13 00:00:20,390][44958] Updated weights for policy 0, policy_version 93880 (0.0007) [2023-10-13 00:00:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13107.2). Total num frames: 192708608. Throughput: 0: 1656.6, 1: 1653.8. Samples: 48182572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:21,443][43579] Avg episode reward: [(0, '282.540'), (1, '284.220')] [2023-10-13 00:00:21,728][44959] Updated weights for policy 1, policy_version 94310 (0.0010) [2023-10-13 00:00:22,101][44959] Updated weights for policy 1, policy_version 94320 (0.0008) [2023-10-13 00:00:22,471][44959] Updated weights for policy 1, policy_version 94330 (0.0008) [2023-10-13 00:00:24,679][44958] Updated weights for policy 0, policy_version 93890 (0.0008) [2023-10-13 00:00:25,047][44958] Updated weights for policy 0, policy_version 93900 (0.0010) [2023-10-13 00:00:25,421][44958] Updated weights for policy 0, policy_version 93910 (0.0008) [2023-10-13 00:00:25,783][44958] Updated weights for policy 0, policy_version 93920 (0.0010) [2023-10-13 00:00:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 192774144. Throughput: 0: 1644.2, 1: 1646.8. Samples: 48201980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:26,443][43579] Avg episode reward: [(0, '283.160'), (1, '281.800')] [2023-10-13 00:00:26,591][44959] Updated weights for policy 1, policy_version 94340 (0.0010) [2023-10-13 00:00:26,965][44959] Updated weights for policy 1, policy_version 94350 (0.0008) [2023-10-13 00:00:27,335][44959] Updated weights for policy 1, policy_version 94360 (0.0008) [2023-10-13 00:00:30,061][44958] Updated weights for policy 0, policy_version 93930 (0.0008) [2023-10-13 00:00:30,437][44958] Updated weights for policy 0, policy_version 93940 (0.0009) [2023-10-13 00:00:30,809][44958] Updated weights for policy 0, policy_version 93950 (0.0008) [2023-10-13 00:00:31,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 192839680. Throughput: 0: 1648.4, 1: 1634.2. Samples: 48221100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:31,443][43579] Avg episode reward: [(0, '283.360'), (1, '280.530')] [2023-10-13 00:00:31,486][44959] Updated weights for policy 1, policy_version 94370 (0.0008) [2023-10-13 00:00:31,860][44959] Updated weights for policy 1, policy_version 94380 (0.0009) [2023-10-13 00:00:32,229][44959] Updated weights for policy 1, policy_version 94390 (0.0009) [2023-10-13 00:00:32,595][44959] Updated weights for policy 1, policy_version 94400 (0.0009) [2023-10-13 00:00:34,806][44958] Updated weights for policy 0, policy_version 93960 (0.0009) [2023-10-13 00:00:35,172][44958] Updated weights for policy 0, policy_version 93970 (0.0009) [2023-10-13 00:00:35,541][44958] Updated weights for policy 0, policy_version 93980 (0.0009) [2023-10-13 00:00:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 192905216. Throughput: 0: 1654.8, 1: 1635.7. Samples: 48231310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:36,444][43579] Avg episode reward: [(0, '287.250'), (1, '275.690')] [2023-10-13 00:00:36,636][44959] Updated weights for policy 1, policy_version 94410 (0.0010) [2023-10-13 00:00:36,997][44959] Updated weights for policy 1, policy_version 94420 (0.0010) [2023-10-13 00:00:37,373][44959] Updated weights for policy 1, policy_version 94430 (0.0008) [2023-10-13 00:00:39,672][44958] Updated weights for policy 0, policy_version 93990 (0.0009) [2023-10-13 00:00:40,044][44958] Updated weights for policy 0, policy_version 94000 (0.0009) [2023-10-13 00:00:40,416][44958] Updated weights for policy 0, policy_version 94010 (0.0008) [2023-10-13 00:00:41,379][44959] Updated weights for policy 1, policy_version 94440 (0.0007) [2023-10-13 00:00:41,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 192970752. Throughput: 0: 1641.2, 1: 1641.6. Samples: 48251068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:41,443][43579] Avg episode reward: [(0, '288.770'), (1, '276.100')] [2023-10-13 00:00:41,743][44959] Updated weights for policy 1, policy_version 94450 (0.0008) [2023-10-13 00:00:42,114][44959] Updated weights for policy 1, policy_version 94460 (0.0007) [2023-10-13 00:00:44,734][44958] Updated weights for policy 0, policy_version 94020 (0.0011) [2023-10-13 00:00:45,126][44958] Updated weights for policy 0, policy_version 94030 (0.0009) [2023-10-13 00:00:45,492][44958] Updated weights for policy 0, policy_version 94040 (0.0008) [2023-10-13 00:00:46,274][44959] Updated weights for policy 1, policy_version 94470 (0.0010) [2023-10-13 00:00:46,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193036288. Throughput: 0: 1646.4, 1: 1644.6. Samples: 48270812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:46,443][43579] Avg episode reward: [(0, '290.770'), (1, '279.560')] [2023-10-13 00:00:46,636][44959] Updated weights for policy 1, policy_version 94480 (0.0008) [2023-10-13 00:00:47,013][44959] Updated weights for policy 1, policy_version 94490 (0.0009) [2023-10-13 00:00:49,709][44958] Updated weights for policy 0, policy_version 94050 (0.0010) [2023-10-13 00:00:50,077][44958] Updated weights for policy 0, policy_version 94060 (0.0010) [2023-10-13 00:00:50,452][44958] Updated weights for policy 0, policy_version 94070 (0.0007) [2023-10-13 00:00:50,814][44958] Updated weights for policy 0, policy_version 94080 (0.0007) [2023-10-13 00:00:51,278][44959] Updated weights for policy 1, policy_version 94500 (0.0008) [2023-10-13 00:00:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193101824. Throughput: 0: 1645.1, 1: 1638.7. Samples: 48280882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:00:51,443][43579] Avg episode reward: [(0, '290.190'), (1, '279.890')] [2023-10-13 00:00:51,657][44959] Updated weights for policy 1, policy_version 94510 (0.0008) [2023-10-13 00:00:52,031][44959] Updated weights for policy 1, policy_version 94520 (0.0008) [2023-10-13 00:00:54,929][44958] Updated weights for policy 0, policy_version 94090 (0.0008) [2023-10-13 00:00:55,305][44958] Updated weights for policy 0, policy_version 94100 (0.0008) [2023-10-13 00:00:55,676][44958] Updated weights for policy 0, policy_version 94110 (0.0009) [2023-10-13 00:00:56,138][44959] Updated weights for policy 1, policy_version 94530 (0.0011) [2023-10-13 00:00:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193167360. Throughput: 0: 1636.7, 1: 1639.8. Samples: 48300460. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:00:56,444][43579] Avg episode reward: [(0, '287.230'), (1, '281.160')] [2023-10-13 00:00:56,514][44959] Updated weights for policy 1, policy_version 94540 (0.0009) [2023-10-13 00:00:56,881][44959] Updated weights for policy 1, policy_version 94550 (0.0008) [2023-10-13 00:00:57,260][44959] Updated weights for policy 1, policy_version 94560 (0.0010) [2023-10-13 00:01:00,089][44958] Updated weights for policy 0, policy_version 94120 (0.0010) [2023-10-13 00:01:00,461][44958] Updated weights for policy 0, policy_version 94130 (0.0009) [2023-10-13 00:01:00,835][44958] Updated weights for policy 0, policy_version 94140 (0.0009) [2023-10-13 00:01:01,403][44959] Updated weights for policy 1, policy_version 94570 (0.0007) [2023-10-13 00:01:01,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193232896. Throughput: 0: 1634.5, 1: 1642.1. Samples: 48319976. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:01,443][43579] Avg episode reward: [(0, '285.850'), (1, '280.540')] [2023-10-13 00:01:01,770][44959] Updated weights for policy 1, policy_version 94580 (0.0007) [2023-10-13 00:01:02,145][44959] Updated weights for policy 1, policy_version 94590 (0.0007) [2023-10-13 00:01:05,071][44958] Updated weights for policy 0, policy_version 94150 (0.0007) [2023-10-13 00:01:05,443][44958] Updated weights for policy 0, policy_version 94160 (0.0007) [2023-10-13 00:01:05,807][44958] Updated weights for policy 0, policy_version 94170 (0.0008) [2023-10-13 00:01:06,262][44959] Updated weights for policy 1, policy_version 94600 (0.0009) [2023-10-13 00:01:06,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 193298432. Throughput: 0: 1637.1, 1: 1644.2. Samples: 48330232. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:06,443][43579] Avg episode reward: [(0, '287.060'), (1, '286.570')] [2023-10-13 00:01:06,621][44959] Updated weights for policy 1, policy_version 94610 (0.0011) [2023-10-13 00:01:06,992][44959] Updated weights for policy 1, policy_version 94620 (0.0009) [2023-10-13 00:01:09,844][44958] Updated weights for policy 0, policy_version 94180 (0.0011) [2023-10-13 00:01:10,215][44958] Updated weights for policy 0, policy_version 94190 (0.0011) [2023-10-13 00:01:10,593][44958] Updated weights for policy 0, policy_version 94200 (0.0010) [2023-10-13 00:01:11,190][44959] Updated weights for policy 1, policy_version 94630 (0.0010) [2023-10-13 00:01:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193363968. Throughput: 0: 1639.9, 1: 1648.7. Samples: 48349966. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:11,444][43579] Avg episode reward: [(0, '290.930'), (1, '289.290')] [2023-10-13 00:01:11,559][44959] Updated weights for policy 1, policy_version 94640 (0.0010) [2023-10-13 00:01:11,930][44959] Updated weights for policy 1, policy_version 94650 (0.0010) [2023-10-13 00:01:14,809][44958] Updated weights for policy 0, policy_version 94210 (0.0010) [2023-10-13 00:01:15,186][44958] Updated weights for policy 0, policy_version 94220 (0.0011) [2023-10-13 00:01:15,552][44958] Updated weights for policy 0, policy_version 94230 (0.0009) [2023-10-13 00:01:15,923][44958] Updated weights for policy 0, policy_version 94240 (0.0010) [2023-10-13 00:01:16,180][44959] Updated weights for policy 1, policy_version 94660 (0.0008) [2023-10-13 00:01:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193429504. Throughput: 0: 1637.1, 1: 1656.3. Samples: 48369306. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:16,444][43579] Avg episode reward: [(0, '283.540'), (1, '288.120')] [2023-10-13 00:01:16,548][44959] Updated weights for policy 1, policy_version 94670 (0.0009) [2023-10-13 00:01:16,915][44959] Updated weights for policy 1, policy_version 94680 (0.0008) [2023-10-13 00:01:19,954][44958] Updated weights for policy 0, policy_version 94250 (0.0007) [2023-10-13 00:01:20,332][44958] Updated weights for policy 0, policy_version 94260 (0.0007) [2023-10-13 00:01:20,702][44958] Updated weights for policy 0, policy_version 94270 (0.0010) [2023-10-13 00:01:20,901][44959] Updated weights for policy 1, policy_version 94690 (0.0007) [2023-10-13 00:01:21,266][44959] Updated weights for policy 1, policy_version 94700 (0.0010) [2023-10-13 00:01:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193495040. Throughput: 0: 1640.4, 1: 1657.3. Samples: 48379706. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:21,444][43579] Avg episode reward: [(0, '279.570'), (1, '284.530')] [2023-10-13 00:01:21,640][44959] Updated weights for policy 1, policy_version 94710 (0.0008) [2023-10-13 00:01:22,009][44959] Updated weights for policy 1, policy_version 94720 (0.0010) [2023-10-13 00:01:24,850][44958] Updated weights for policy 0, policy_version 94280 (0.0009) [2023-10-13 00:01:25,222][44958] Updated weights for policy 0, policy_version 94290 (0.0008) [2023-10-13 00:01:25,585][44958] Updated weights for policy 0, policy_version 94300 (0.0010) [2023-10-13 00:01:26,279][44959] Updated weights for policy 1, policy_version 94730 (0.0010) [2023-10-13 00:01:26,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193560576. Throughput: 0: 1640.5, 1: 1655.0. Samples: 48399366. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:26,443][43579] Avg episode reward: [(0, '280.410'), (1, '281.560')] [2023-10-13 00:01:26,642][44959] Updated weights for policy 1, policy_version 94740 (0.0008) [2023-10-13 00:01:27,020][44959] Updated weights for policy 1, policy_version 94750 (0.0007) [2023-10-13 00:01:29,804][44958] Updated weights for policy 0, policy_version 94310 (0.0010) [2023-10-13 00:01:30,189][44958] Updated weights for policy 0, policy_version 94320 (0.0009) [2023-10-13 00:01:30,550][44958] Updated weights for policy 0, policy_version 94330 (0.0009) [2023-10-13 00:01:31,080][44959] Updated weights for policy 1, policy_version 94760 (0.0008) [2023-10-13 00:01:31,442][44959] Updated weights for policy 1, policy_version 94770 (0.0010) [2023-10-13 00:01:31,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193626112. Throughput: 0: 1641.9, 1: 1645.1. Samples: 48418726. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:31,443][43579] Avg episode reward: [(0, '279.450'), (1, '277.730')] [2023-10-13 00:01:31,810][44959] Updated weights for policy 1, policy_version 94780 (0.0011) [2023-10-13 00:01:34,772][44958] Updated weights for policy 0, policy_version 94340 (0.0009) [2023-10-13 00:01:35,140][44958] Updated weights for policy 0, policy_version 94350 (0.0009) [2023-10-13 00:01:35,510][44958] Updated weights for policy 0, policy_version 94360 (0.0009) [2023-10-13 00:01:36,022][44959] Updated weights for policy 1, policy_version 94790 (0.0009) [2023-10-13 00:01:36,418][44959] Updated weights for policy 1, policy_version 94800 (0.0008) [2023-10-13 00:01:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193691648. Throughput: 0: 1641.3, 1: 1650.8. Samples: 48429026. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:36,443][43579] Avg episode reward: [(0, '276.700'), (1, '267.410')] [2023-10-13 00:01:36,787][44959] Updated weights for policy 1, policy_version 94810 (0.0007) [2023-10-13 00:01:39,645][44958] Updated weights for policy 0, policy_version 94370 (0.0009) [2023-10-13 00:01:40,012][44958] Updated weights for policy 0, policy_version 94380 (0.0008) [2023-10-13 00:01:40,388][44958] Updated weights for policy 0, policy_version 94390 (0.0008) [2023-10-13 00:01:40,755][44958] Updated weights for policy 0, policy_version 94400 (0.0010) [2023-10-13 00:01:40,902][44959] Updated weights for policy 1, policy_version 94820 (0.0008) [2023-10-13 00:01:41,266][44959] Updated weights for policy 1, policy_version 94830 (0.0008) [2023-10-13 00:01:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193757184. Throughput: 0: 1639.9, 1: 1652.0. Samples: 48448594. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:41,443][43579] Avg episode reward: [(0, '278.980'), (1, '268.550')] [2023-10-13 00:01:41,627][44959] Updated weights for policy 1, policy_version 94840 (0.0009) [2023-10-13 00:01:45,000][44958] Updated weights for policy 0, policy_version 94410 (0.0008) [2023-10-13 00:01:45,377][44958] Updated weights for policy 0, policy_version 94420 (0.0009) [2023-10-13 00:01:45,531][44959] Updated weights for policy 1, policy_version 94850 (0.0010) [2023-10-13 00:01:45,757][44958] Updated weights for policy 0, policy_version 94430 (0.0009) [2023-10-13 00:01:45,900][44959] Updated weights for policy 1, policy_version 94860 (0.0009) [2023-10-13 00:01:46,263][44959] Updated weights for policy 1, policy_version 94870 (0.0009) [2023-10-13 00:01:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193822720. Throughput: 0: 1643.6, 1: 1645.0. Samples: 48467962. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:46,443][43579] Avg episode reward: [(0, '280.360'), (1, '272.720')] [2023-10-13 00:01:46,452][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000094432_96698368.pth... [2023-10-13 00:01:46,487][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000092896_95125504.pth [2023-10-13 00:01:46,627][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000094880_97157120.pth... [2023-10-13 00:01:46,628][44959] Updated weights for policy 1, policy_version 94880 (0.0009) [2023-10-13 00:01:46,668][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000093312_95551488.pth [2023-10-13 00:01:50,004][44958] Updated weights for policy 0, policy_version 94440 (0.0008) [2023-10-13 00:01:50,381][44958] Updated weights for policy 0, policy_version 94450 (0.0009) [2023-10-13 00:01:50,751][44958] Updated weights for policy 0, policy_version 94460 (0.0007) [2023-10-13 00:01:50,879][44959] Updated weights for policy 1, policy_version 94890 (0.0008) [2023-10-13 00:01:51,255][44959] Updated weights for policy 1, policy_version 94900 (0.0009) [2023-10-13 00:01:51,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193888256. Throughput: 0: 1639.6, 1: 1656.8. Samples: 48478566. Policy #0 lag: (min: 31.0, avg: 32.5, max: 59.0) [2023-10-13 00:01:51,443][43579] Avg episode reward: [(0, '283.790'), (1, '278.640')] [2023-10-13 00:01:51,625][44959] Updated weights for policy 1, policy_version 94910 (0.0009) [2023-10-13 00:01:54,937][44958] Updated weights for policy 0, policy_version 94470 (0.0008) [2023-10-13 00:01:55,310][44958] Updated weights for policy 0, policy_version 94480 (0.0008) [2023-10-13 00:01:55,677][44958] Updated weights for policy 0, policy_version 94490 (0.0011) [2023-10-13 00:01:55,747][44959] Updated weights for policy 1, policy_version 94920 (0.0008) [2023-10-13 00:01:56,124][44959] Updated weights for policy 1, policy_version 94930 (0.0009) [2023-10-13 00:01:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 193953792. Throughput: 0: 1635.5, 1: 1660.5. Samples: 48498286. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:01:56,444][43579] Avg episode reward: [(0, '286.790'), (1, '274.990')] [2023-10-13 00:01:56,486][44959] Updated weights for policy 1, policy_version 94940 (0.0007) [2023-10-13 00:01:59,742][44958] Updated weights for policy 0, policy_version 94500 (0.0010) [2023-10-13 00:02:00,121][44958] Updated weights for policy 0, policy_version 94510 (0.0009) [2023-10-13 00:02:00,485][44958] Updated weights for policy 0, policy_version 94520 (0.0008) [2023-10-13 00:02:00,629][44959] Updated weights for policy 1, policy_version 94950 (0.0008) [2023-10-13 00:02:00,997][44959] Updated weights for policy 1, policy_version 94960 (0.0009) [2023-10-13 00:02:01,358][44959] Updated weights for policy 1, policy_version 94970 (0.0009) [2023-10-13 00:02:01,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194019328. Throughput: 0: 1642.2, 1: 1644.4. Samples: 48517206. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:01,443][43579] Avg episode reward: [(0, '283.750'), (1, '277.350')] [2023-10-13 00:02:04,707][44958] Updated weights for policy 0, policy_version 94530 (0.0008) [2023-10-13 00:02:05,071][44958] Updated weights for policy 0, policy_version 94540 (0.0008) [2023-10-13 00:02:05,445][44958] Updated weights for policy 0, policy_version 94550 (0.0010) [2023-10-13 00:02:05,542][44959] Updated weights for policy 1, policy_version 94980 (0.0008) [2023-10-13 00:02:05,813][44958] Updated weights for policy 0, policy_version 94560 (0.0009) [2023-10-13 00:02:05,908][44959] Updated weights for policy 1, policy_version 94990 (0.0009) [2023-10-13 00:02:06,273][44959] Updated weights for policy 1, policy_version 95000 (0.0007) [2023-10-13 00:02:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194084864. Throughput: 0: 1634.9, 1: 1655.7. Samples: 48527784. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:06,443][43579] Avg episode reward: [(0, '283.590'), (1, '287.720')] [2023-10-13 00:02:09,896][44958] Updated weights for policy 0, policy_version 94570 (0.0009) [2023-10-13 00:02:10,273][44958] Updated weights for policy 0, policy_version 94580 (0.0009) [2023-10-13 00:02:10,559][44959] Updated weights for policy 1, policy_version 95010 (0.0009) [2023-10-13 00:02:10,643][44958] Updated weights for policy 0, policy_version 94590 (0.0009) [2023-10-13 00:02:10,928][44959] Updated weights for policy 1, policy_version 95020 (0.0009) [2023-10-13 00:02:11,299][44959] Updated weights for policy 1, policy_version 95030 (0.0009) [2023-10-13 00:02:11,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194150400. Throughput: 0: 1637.2, 1: 1653.6. Samples: 48547452. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:11,443][43579] Avg episode reward: [(0, '284.090'), (1, '282.440')] [2023-10-13 00:02:11,667][44959] Updated weights for policy 1, policy_version 95040 (0.0007) [2023-10-13 00:02:15,013][44958] Updated weights for policy 0, policy_version 94600 (0.0007) [2023-10-13 00:02:15,395][44958] Updated weights for policy 0, policy_version 94610 (0.0009) [2023-10-13 00:02:15,771][44958] Updated weights for policy 0, policy_version 94620 (0.0007) [2023-10-13 00:02:15,792][44959] Updated weights for policy 1, policy_version 95050 (0.0008) [2023-10-13 00:02:16,160][44959] Updated weights for policy 1, policy_version 95060 (0.0008) [2023-10-13 00:02:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194215936. Throughput: 0: 1636.4, 1: 1647.8. Samples: 48566516. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:16,443][43579] Avg episode reward: [(0, '281.230'), (1, '283.600')] [2023-10-13 00:02:16,530][44959] Updated weights for policy 1, policy_version 95070 (0.0008) [2023-10-13 00:02:19,923][44958] Updated weights for policy 0, policy_version 94630 (0.0007) [2023-10-13 00:02:20,297][44958] Updated weights for policy 0, policy_version 94640 (0.0009) [2023-10-13 00:02:20,674][44958] Updated weights for policy 0, policy_version 94650 (0.0009) [2023-10-13 00:02:20,814][44959] Updated weights for policy 1, policy_version 95080 (0.0008) [2023-10-13 00:02:21,183][44959] Updated weights for policy 1, policy_version 95090 (0.0008) [2023-10-13 00:02:21,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194281472. Throughput: 0: 1637.4, 1: 1652.8. Samples: 48577086. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:21,444][43579] Avg episode reward: [(0, '276.880'), (1, '281.260')] [2023-10-13 00:02:21,558][44959] Updated weights for policy 1, policy_version 95100 (0.0010) [2023-10-13 00:02:24,791][44958] Updated weights for policy 0, policy_version 94660 (0.0009) [2023-10-13 00:02:25,171][44958] Updated weights for policy 0, policy_version 94670 (0.0009) [2023-10-13 00:02:25,548][44958] Updated weights for policy 0, policy_version 94680 (0.0009) [2023-10-13 00:02:25,691][44959] Updated weights for policy 1, policy_version 95110 (0.0010) [2023-10-13 00:02:26,050][44959] Updated weights for policy 1, policy_version 95120 (0.0008) [2023-10-13 00:02:26,427][44959] Updated weights for policy 1, policy_version 95130 (0.0008) [2023-10-13 00:02:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194347008. Throughput: 0: 1641.2, 1: 1650.3. Samples: 48596710. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:26,443][43579] Avg episode reward: [(0, '278.380'), (1, '279.360')] [2023-10-13 00:02:29,716][44958] Updated weights for policy 0, policy_version 94690 (0.0009) [2023-10-13 00:02:30,093][44958] Updated weights for policy 0, policy_version 94700 (0.0008) [2023-10-13 00:02:30,462][44958] Updated weights for policy 0, policy_version 94710 (0.0009) [2023-10-13 00:02:30,495][44959] Updated weights for policy 1, policy_version 95140 (0.0008) [2023-10-13 00:02:30,829][44958] Updated weights for policy 0, policy_version 94720 (0.0009) [2023-10-13 00:02:30,867][44959] Updated weights for policy 1, policy_version 95150 (0.0007) [2023-10-13 00:02:31,241][44959] Updated weights for policy 1, policy_version 95160 (0.0009) [2023-10-13 00:02:31,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194412544. Throughput: 0: 1636.3, 1: 1642.7. Samples: 48615514. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:31,443][43579] Avg episode reward: [(0, '284.890'), (1, '277.090')] [2023-10-13 00:02:35,125][44958] Updated weights for policy 0, policy_version 94730 (0.0007) [2023-10-13 00:02:35,310][44959] Updated weights for policy 1, policy_version 95170 (0.0009) [2023-10-13 00:02:35,494][44958] Updated weights for policy 0, policy_version 94740 (0.0008) [2023-10-13 00:02:35,684][44959] Updated weights for policy 1, policy_version 95180 (0.0008) [2023-10-13 00:02:35,864][44958] Updated weights for policy 0, policy_version 94750 (0.0009) [2023-10-13 00:02:36,042][44959] Updated weights for policy 1, policy_version 95190 (0.0007) [2023-10-13 00:02:36,403][44959] Updated weights for policy 1, policy_version 95200 (0.0008) [2023-10-13 00:02:36,443][43579] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 194510848. Throughput: 0: 1636.3, 1: 1647.3. Samples: 48626326. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:36,444][43579] Avg episode reward: [(0, '285.420'), (1, '278.080')] [2023-10-13 00:02:39,829][44958] Updated weights for policy 0, policy_version 94760 (0.0008) [2023-10-13 00:02:40,207][44958] Updated weights for policy 0, policy_version 94770 (0.0007) [2023-10-13 00:02:40,577][44958] Updated weights for policy 0, policy_version 94780 (0.0007) [2023-10-13 00:02:40,790][44959] Updated weights for policy 1, policy_version 95210 (0.0009) [2023-10-13 00:02:41,159][44959] Updated weights for policy 1, policy_version 95220 (0.0009) [2023-10-13 00:02:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194543616. Throughput: 0: 1637.6, 1: 1644.5. Samples: 48645982. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:41,443][43579] Avg episode reward: [(0, '286.880'), (1, '273.370')] [2023-10-13 00:02:41,522][44959] Updated weights for policy 1, policy_version 95230 (0.0008) [2023-10-13 00:02:44,643][44958] Updated weights for policy 0, policy_version 94790 (0.0009) [2023-10-13 00:02:45,014][44958] Updated weights for policy 0, policy_version 94800 (0.0008) [2023-10-13 00:02:45,380][44958] Updated weights for policy 0, policy_version 94810 (0.0008) [2023-10-13 00:02:45,704][44959] Updated weights for policy 1, policy_version 95240 (0.0008) [2023-10-13 00:02:46,079][44959] Updated weights for policy 1, policy_version 95250 (0.0011) [2023-10-13 00:02:46,443][43579] Fps is (10 sec: 9830.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194609152. Throughput: 0: 1642.8, 1: 1642.0. Samples: 48665026. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:46,443][43579] Avg episode reward: [(0, '290.350'), (1, '275.750')] [2023-10-13 00:02:46,450][44959] Updated weights for policy 1, policy_version 95260 (0.0011) [2023-10-13 00:02:49,582][44958] Updated weights for policy 0, policy_version 94820 (0.0009) [2023-10-13 00:02:49,957][44958] Updated weights for policy 0, policy_version 94830 (0.0007) [2023-10-13 00:02:50,333][44958] Updated weights for policy 0, policy_version 94840 (0.0009) [2023-10-13 00:02:50,469][44959] Updated weights for policy 1, policy_version 95270 (0.0007) [2023-10-13 00:02:50,838][44959] Updated weights for policy 1, policy_version 95280 (0.0009) [2023-10-13 00:02:51,208][44959] Updated weights for policy 1, policy_version 95290 (0.0008) [2023-10-13 00:02:51,443][43579] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 194707456. Throughput: 0: 1642.4, 1: 1645.1. Samples: 48675722. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-13 00:02:51,444][43579] Avg episode reward: [(0, '290.860'), (1, '278.180')] [2023-10-13 00:02:54,511][44958] Updated weights for policy 0, policy_version 94850 (0.0008) [2023-10-13 00:02:54,873][44958] Updated weights for policy 0, policy_version 94860 (0.0008) [2023-10-13 00:02:55,244][44958] Updated weights for policy 0, policy_version 94870 (0.0009) [2023-10-13 00:02:55,497][44959] Updated weights for policy 1, policy_version 95300 (0.0008) [2023-10-13 00:02:55,613][44958] Updated weights for policy 0, policy_version 94880 (0.0009) [2023-10-13 00:02:55,864][44959] Updated weights for policy 1, policy_version 95310 (0.0008) [2023-10-13 00:02:56,236][44959] Updated weights for policy 1, policy_version 95320 (0.0008) [2023-10-13 00:02:56,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 194740224. Throughput: 0: 1640.7, 1: 1644.6. Samples: 48695290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:02:56,443][43579] Avg episode reward: [(0, '283.710'), (1, '276.350')] [2023-10-13 00:02:59,953][44958] Updated weights for policy 0, policy_version 94890 (0.0008) [2023-10-13 00:03:00,311][44959] Updated weights for policy 1, policy_version 95330 (0.0009) [2023-10-13 00:03:00,326][44958] Updated weights for policy 0, policy_version 94900 (0.0008) [2023-10-13 00:03:00,678][44959] Updated weights for policy 1, policy_version 95340 (0.0007) [2023-10-13 00:03:00,698][44958] Updated weights for policy 0, policy_version 94910 (0.0007) [2023-10-13 00:03:01,046][44959] Updated weights for policy 1, policy_version 95350 (0.0010) [2023-10-13 00:03:01,413][44959] Updated weights for policy 1, policy_version 95360 (0.0009) [2023-10-13 00:03:01,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 194838528. Throughput: 0: 1638.3, 1: 1636.9. Samples: 48713900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:01,443][43579] Avg episode reward: [(0, '282.700'), (1, '276.680')] [2023-10-13 00:03:05,028][44958] Updated weights for policy 0, policy_version 94920 (0.0007) [2023-10-13 00:03:05,398][44958] Updated weights for policy 0, policy_version 94930 (0.0009) [2023-10-13 00:03:05,455][44959] Updated weights for policy 1, policy_version 95370 (0.0007) [2023-10-13 00:03:05,756][44958] Updated weights for policy 0, policy_version 94940 (0.0009) [2023-10-13 00:03:05,827][44959] Updated weights for policy 1, policy_version 95380 (0.0008) [2023-10-13 00:03:06,183][44959] Updated weights for policy 1, policy_version 95390 (0.0008) [2023-10-13 00:03:06,442][43579] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 194904064. Throughput: 0: 1635.8, 1: 1649.0. Samples: 48724902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:06,443][43579] Avg episode reward: [(0, '282.280'), (1, '273.770')] [2023-10-13 00:03:09,847][44958] Updated weights for policy 0, policy_version 94950 (0.0007) [2023-10-13 00:03:10,219][44958] Updated weights for policy 0, policy_version 94960 (0.0007) [2023-10-13 00:03:10,466][44959] Updated weights for policy 1, policy_version 95400 (0.0008) [2023-10-13 00:03:10,590][44958] Updated weights for policy 0, policy_version 94970 (0.0009) [2023-10-13 00:03:10,845][44959] Updated weights for policy 1, policy_version 95410 (0.0009) [2023-10-13 00:03:11,212][44959] Updated weights for policy 1, policy_version 95420 (0.0010) [2023-10-13 00:03:11,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 194969600. Throughput: 0: 1638.5, 1: 1650.9. Samples: 48744736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:11,443][43579] Avg episode reward: [(0, '286.330'), (1, '277.810')] [2023-10-13 00:03:14,644][44958] Updated weights for policy 0, policy_version 94980 (0.0007) [2023-10-13 00:03:15,011][44958] Updated weights for policy 0, policy_version 94990 (0.0007) [2023-10-13 00:03:15,348][44959] Updated weights for policy 1, policy_version 95430 (0.0008) [2023-10-13 00:03:15,382][44958] Updated weights for policy 0, policy_version 95000 (0.0007) [2023-10-13 00:03:15,716][44959] Updated weights for policy 1, policy_version 95440 (0.0008) [2023-10-13 00:03:16,079][44959] Updated weights for policy 1, policy_version 95450 (0.0007) [2023-10-13 00:03:16,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 195035136. Throughput: 0: 1643.6, 1: 1639.0. Samples: 48763232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:16,443][43579] Avg episode reward: [(0, '283.750'), (1, '277.590')] [2023-10-13 00:03:19,507][44958] Updated weights for policy 0, policy_version 95010 (0.0008) [2023-10-13 00:03:19,870][44958] Updated weights for policy 0, policy_version 95020 (0.0008) [2023-10-13 00:03:20,242][44958] Updated weights for policy 0, policy_version 95030 (0.0010) [2023-10-13 00:03:20,295][44959] Updated weights for policy 1, policy_version 95460 (0.0008) [2023-10-13 00:03:20,608][44958] Updated weights for policy 0, policy_version 95040 (0.0009) [2023-10-13 00:03:20,669][44959] Updated weights for policy 1, policy_version 95470 (0.0009) [2023-10-13 00:03:21,036][44959] Updated weights for policy 1, policy_version 95480 (0.0009) [2023-10-13 00:03:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 195100672. Throughput: 0: 1645.1, 1: 1645.7. Samples: 48774414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:21,443][43579] Avg episode reward: [(0, '283.880'), (1, '277.120')] [2023-10-13 00:03:24,787][44958] Updated weights for policy 0, policy_version 95050 (0.0011) [2023-10-13 00:03:25,110][44959] Updated weights for policy 1, policy_version 95490 (0.0008) [2023-10-13 00:03:25,156][44958] Updated weights for policy 0, policy_version 95060 (0.0008) [2023-10-13 00:03:25,481][44959] Updated weights for policy 1, policy_version 95500 (0.0007) [2023-10-13 00:03:25,535][44958] Updated weights for policy 0, policy_version 95070 (0.0009) [2023-10-13 00:03:25,852][44959] Updated weights for policy 1, policy_version 95510 (0.0008) [2023-10-13 00:03:26,214][44959] Updated weights for policy 1, policy_version 95520 (0.0008) [2023-10-13 00:03:26,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 195166208. Throughput: 0: 1642.8, 1: 1646.8. Samples: 48794012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:26,443][43579] Avg episode reward: [(0, '291.980'), (1, '277.120')] [2023-10-13 00:03:29,896][44958] Updated weights for policy 0, policy_version 95080 (0.0009) [2023-10-13 00:03:30,265][44958] Updated weights for policy 0, policy_version 95090 (0.0008) [2023-10-13 00:03:30,370][44959] Updated weights for policy 1, policy_version 95530 (0.0008) [2023-10-13 00:03:30,644][44958] Updated weights for policy 0, policy_version 95100 (0.0009) [2023-10-13 00:03:30,743][44959] Updated weights for policy 1, policy_version 95540 (0.0009) [2023-10-13 00:03:31,107][44959] Updated weights for policy 1, policy_version 95550 (0.0011) [2023-10-13 00:03:31,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 195231744. Throughput: 0: 1633.8, 1: 1643.0. Samples: 48812480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:31,444][43579] Avg episode reward: [(0, '290.740'), (1, '278.570')] [2023-10-13 00:03:34,791][44958] Updated weights for policy 0, policy_version 95110 (0.0009) [2023-10-13 00:03:35,163][44958] Updated weights for policy 0, policy_version 95120 (0.0009) [2023-10-13 00:03:35,424][44959] Updated weights for policy 1, policy_version 95560 (0.0007) [2023-10-13 00:03:35,537][44958] Updated weights for policy 0, policy_version 95130 (0.0009) [2023-10-13 00:03:35,791][44959] Updated weights for policy 1, policy_version 95570 (0.0009) [2023-10-13 00:03:36,155][44959] Updated weights for policy 1, policy_version 95580 (0.0008) [2023-10-13 00:03:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 195297280. Throughput: 0: 1637.1, 1: 1650.3. Samples: 48823654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:36,444][43579] Avg episode reward: [(0, '290.280'), (1, '284.360')] [2023-10-13 00:03:39,609][44958] Updated weights for policy 0, policy_version 95140 (0.0010) [2023-10-13 00:03:39,981][44958] Updated weights for policy 0, policy_version 95150 (0.0008) [2023-10-13 00:03:40,343][44958] Updated weights for policy 0, policy_version 95160 (0.0007) [2023-10-13 00:03:40,543][44959] Updated weights for policy 1, policy_version 95590 (0.0007) [2023-10-13 00:03:40,903][44959] Updated weights for policy 1, policy_version 95600 (0.0007) [2023-10-13 00:03:41,274][44959] Updated weights for policy 1, policy_version 95610 (0.0007) [2023-10-13 00:03:41,442][43579] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 195330048. Throughput: 0: 1639.6, 1: 1640.3. Samples: 48842884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:41,443][43579] Avg episode reward: [(0, '286.740'), (1, '285.420')] [2023-10-13 00:03:44,818][44958] Updated weights for policy 0, policy_version 95170 (0.0009) [2023-10-13 00:03:45,229][44958] Updated weights for policy 0, policy_version 95180 (0.0008) [2023-10-13 00:03:45,366][44959] Updated weights for policy 1, policy_version 95620 (0.0007) [2023-10-13 00:03:45,585][44958] Updated weights for policy 0, policy_version 95190 (0.0008) [2023-10-13 00:03:45,727][44959] Updated weights for policy 1, policy_version 95630 (0.0010) [2023-10-13 00:03:45,952][44958] Updated weights for policy 0, policy_version 95200 (0.0009) [2023-10-13 00:03:46,094][44959] Updated weights for policy 1, policy_version 95640 (0.0008) [2023-10-13 00:03:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 195428352. Throughput: 0: 1635.3, 1: 1647.0. Samples: 48861604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:46,444][43579] Avg episode reward: [(0, '292.420'), (1, '283.300')] [2023-10-13 00:03:46,453][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000095200_97484800.pth... [2023-10-13 00:03:46,453][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000095648_97943552.pth... [2023-10-13 00:03:46,492][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000093664_95911936.pth [2023-10-13 00:03:46,494][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000094112_96370688.pth [2023-10-13 00:03:49,937][44958] Updated weights for policy 0, policy_version 95210 (0.0008) [2023-10-13 00:03:50,285][44959] Updated weights for policy 1, policy_version 95650 (0.0007) [2023-10-13 00:03:50,310][44958] Updated weights for policy 0, policy_version 95220 (0.0008) [2023-10-13 00:03:50,656][44959] Updated weights for policy 1, policy_version 95660 (0.0008) [2023-10-13 00:03:50,684][44958] Updated weights for policy 0, policy_version 95230 (0.0009) [2023-10-13 00:03:51,021][44959] Updated weights for policy 1, policy_version 95670 (0.0008) [2023-10-13 00:03:51,390][44959] Updated weights for policy 1, policy_version 95680 (0.0008) [2023-10-13 00:03:51,442][43579] Fps is (10 sec: 16383.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 195493888. Throughput: 0: 1636.4, 1: 1640.3. Samples: 48872350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:03:51,443][43579] Avg episode reward: [(0, '289.380'), (1, '283.460')] [2023-10-13 00:03:54,892][44958] Updated weights for policy 0, policy_version 95240 (0.0008) [2023-10-13 00:03:55,272][44958] Updated weights for policy 0, policy_version 95250 (0.0008) [2023-10-13 00:03:55,641][44958] Updated weights for policy 0, policy_version 95260 (0.0008) [2023-10-13 00:03:55,665][44959] Updated weights for policy 1, policy_version 95690 (0.0010) [2023-10-13 00:03:56,030][44959] Updated weights for policy 1, policy_version 95700 (0.0009) [2023-10-13 00:03:56,400][44959] Updated weights for policy 1, policy_version 95710 (0.0008) [2023-10-13 00:03:56,442][43579] Fps is (10 sec: 9830.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 195526656. Throughput: 0: 1631.4, 1: 1641.9. Samples: 48892032. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:03:56,443][43579] Avg episode reward: [(0, '288.530'), (1, '285.440')] [2023-10-13 00:03:59,924][44958] Updated weights for policy 0, policy_version 95270 (0.0007) [2023-10-13 00:04:00,298][44958] Updated weights for policy 0, policy_version 95280 (0.0009) [2023-10-13 00:04:00,500][44959] Updated weights for policy 1, policy_version 95720 (0.0010) [2023-10-13 00:04:00,674][44958] Updated weights for policy 0, policy_version 95290 (0.0007) [2023-10-13 00:04:00,868][44959] Updated weights for policy 1, policy_version 95730 (0.0009) [2023-10-13 00:04:01,243][44959] Updated weights for policy 1, policy_version 95740 (0.0010) [2023-10-13 00:04:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 195624960. Throughput: 0: 1632.4, 1: 1645.5. Samples: 48910738. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:01,443][43579] Avg episode reward: [(0, '288.520'), (1, '284.230')] [2023-10-13 00:04:04,637][44958] Updated weights for policy 0, policy_version 95300 (0.0008) [2023-10-13 00:04:05,003][44958] Updated weights for policy 0, policy_version 95310 (0.0008) [2023-10-13 00:04:05,384][44958] Updated weights for policy 0, policy_version 95320 (0.0009) [2023-10-13 00:04:05,440][44959] Updated weights for policy 1, policy_version 95750 (0.0010) [2023-10-13 00:04:05,808][44959] Updated weights for policy 1, policy_version 95760 (0.0009) [2023-10-13 00:04:06,174][44959] Updated weights for policy 1, policy_version 95770 (0.0009) [2023-10-13 00:04:06,442][43579] Fps is (10 sec: 16384.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195690496. Throughput: 0: 1635.1, 1: 1638.9. Samples: 48921744. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:06,443][43579] Avg episode reward: [(0, '288.710'), (1, '277.700')] [2023-10-13 00:04:09,659][44958] Updated weights for policy 0, policy_version 95330 (0.0008) [2023-10-13 00:04:10,029][44958] Updated weights for policy 0, policy_version 95340 (0.0007) [2023-10-13 00:04:10,377][44959] Updated weights for policy 1, policy_version 95780 (0.0009) [2023-10-13 00:04:10,407][44958] Updated weights for policy 0, policy_version 95350 (0.0007) [2023-10-13 00:04:10,749][44959] Updated weights for policy 1, policy_version 95790 (0.0008) [2023-10-13 00:04:10,776][44958] Updated weights for policy 0, policy_version 95360 (0.0007) [2023-10-13 00:04:11,127][44959] Updated weights for policy 1, policy_version 95800 (0.0008) [2023-10-13 00:04:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 195756032. Throughput: 0: 1636.1, 1: 1635.7. Samples: 48941244. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:11,443][43579] Avg episode reward: [(0, '291.480'), (1, '276.240')] [2023-10-13 00:04:15,023][44958] Updated weights for policy 0, policy_version 95370 (0.0009) [2023-10-13 00:04:15,168][44959] Updated weights for policy 1, policy_version 95810 (0.0008) [2023-10-13 00:04:15,395][44958] Updated weights for policy 0, policy_version 95380 (0.0009) [2023-10-13 00:04:15,536][44959] Updated weights for policy 1, policy_version 95820 (0.0008) [2023-10-13 00:04:15,766][44958] Updated weights for policy 0, policy_version 95390 (0.0009) [2023-10-13 00:04:15,906][44959] Updated weights for policy 1, policy_version 95830 (0.0007) [2023-10-13 00:04:16,272][44959] Updated weights for policy 1, policy_version 95840 (0.0007) [2023-10-13 00:04:16,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195821568. Throughput: 0: 1636.4, 1: 1639.3. Samples: 48959888. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:16,443][43579] Avg episode reward: [(0, '287.360'), (1, '280.280')] [2023-10-13 00:04:19,614][44958] Updated weights for policy 0, policy_version 95400 (0.0008) [2023-10-13 00:04:19,990][44958] Updated weights for policy 0, policy_version 95410 (0.0009) [2023-10-13 00:04:20,365][44958] Updated weights for policy 0, policy_version 95420 (0.0009) [2023-10-13 00:04:20,373][44959] Updated weights for policy 1, policy_version 95850 (0.0008) [2023-10-13 00:04:20,735][44959] Updated weights for policy 1, policy_version 95860 (0.0011) [2023-10-13 00:04:21,112][44959] Updated weights for policy 1, policy_version 95870 (0.0009) [2023-10-13 00:04:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 195887104. Throughput: 0: 1639.9, 1: 1634.4. Samples: 48970998. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:21,443][43579] Avg episode reward: [(0, '289.460'), (1, '284.840')] [2023-10-13 00:04:24,632][44958] Updated weights for policy 0, policy_version 95430 (0.0007) [2023-10-13 00:04:25,012][44958] Updated weights for policy 0, policy_version 95440 (0.0007) [2023-10-13 00:04:25,376][44959] Updated weights for policy 1, policy_version 95880 (0.0008) [2023-10-13 00:04:25,389][44958] Updated weights for policy 0, policy_version 95450 (0.0008) [2023-10-13 00:04:25,734][44959] Updated weights for policy 1, policy_version 95890 (0.0009) [2023-10-13 00:04:26,098][44959] Updated weights for policy 1, policy_version 95900 (0.0010) [2023-10-13 00:04:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 195952640. Throughput: 0: 1638.8, 1: 1642.9. Samples: 48990562. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:26,443][43579] Avg episode reward: [(0, '286.570'), (1, '283.720')] [2023-10-13 00:04:29,694][44958] Updated weights for policy 0, policy_version 95460 (0.0009) [2023-10-13 00:04:30,087][44958] Updated weights for policy 0, policy_version 95470 (0.0010) [2023-10-13 00:04:30,230][44959] Updated weights for policy 1, policy_version 95910 (0.0008) [2023-10-13 00:04:30,449][44958] Updated weights for policy 0, policy_version 95480 (0.0009) [2023-10-13 00:04:30,602][44959] Updated weights for policy 1, policy_version 95920 (0.0009) [2023-10-13 00:04:30,966][44959] Updated weights for policy 1, policy_version 95930 (0.0008) [2023-10-13 00:04:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196018176. Throughput: 0: 1642.1, 1: 1632.8. Samples: 49008976. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:31,443][43579] Avg episode reward: [(0, '286.220'), (1, '285.090')] [2023-10-13 00:04:34,686][44958] Updated weights for policy 0, policy_version 95490 (0.0009) [2023-10-13 00:04:35,066][44958] Updated weights for policy 0, policy_version 95500 (0.0009) [2023-10-13 00:04:35,116][44959] Updated weights for policy 1, policy_version 95940 (0.0008) [2023-10-13 00:04:35,428][44958] Updated weights for policy 0, policy_version 95510 (0.0008) [2023-10-13 00:04:35,483][44959] Updated weights for policy 1, policy_version 95950 (0.0008) [2023-10-13 00:04:35,802][44958] Updated weights for policy 0, policy_version 95520 (0.0009) [2023-10-13 00:04:35,850][44959] Updated weights for policy 1, policy_version 95960 (0.0009) [2023-10-13 00:04:36,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 196083712. Throughput: 0: 1643.1, 1: 1640.8. Samples: 49020124. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:36,444][43579] Avg episode reward: [(0, '284.160'), (1, '289.100')] [2023-10-13 00:04:39,880][44958] Updated weights for policy 0, policy_version 95530 (0.0010) [2023-10-13 00:04:40,249][44958] Updated weights for policy 0, policy_version 95540 (0.0008) [2023-10-13 00:04:40,274][44959] Updated weights for policy 1, policy_version 95970 (0.0009) [2023-10-13 00:04:40,622][44958] Updated weights for policy 0, policy_version 95550 (0.0007) [2023-10-13 00:04:40,687][44959] Updated weights for policy 1, policy_version 95980 (0.0008) [2023-10-13 00:04:41,063][44959] Updated weights for policy 1, policy_version 95990 (0.0007) [2023-10-13 00:04:41,425][44959] Updated weights for policy 1, policy_version 96000 (0.0007) [2023-10-13 00:04:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 196149248. Throughput: 0: 1640.9, 1: 1639.2. Samples: 49039636. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:41,443][43579] Avg episode reward: [(0, '286.160'), (1, '285.010')] [2023-10-13 00:04:44,876][44958] Updated weights for policy 0, policy_version 95560 (0.0009) [2023-10-13 00:04:45,256][44958] Updated weights for policy 0, policy_version 95570 (0.0010) [2023-10-13 00:04:45,497][44959] Updated weights for policy 1, policy_version 96010 (0.0009) [2023-10-13 00:04:45,624][44958] Updated weights for policy 0, policy_version 95580 (0.0009) [2023-10-13 00:04:45,862][44959] Updated weights for policy 1, policy_version 96020 (0.0009) [2023-10-13 00:04:46,229][44959] Updated weights for policy 1, policy_version 96030 (0.0007) [2023-10-13 00:04:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 196214784. Throughput: 0: 1639.6, 1: 1638.1. Samples: 49058236. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:46,444][43579] Avg episode reward: [(0, '290.660'), (1, '285.310')] [2023-10-13 00:04:49,727][44958] Updated weights for policy 0, policy_version 95590 (0.0009) [2023-10-13 00:04:50,103][44958] Updated weights for policy 0, policy_version 95600 (0.0008) [2023-10-13 00:04:50,464][44959] Updated weights for policy 1, policy_version 96040 (0.0008) [2023-10-13 00:04:50,466][44958] Updated weights for policy 0, policy_version 95610 (0.0007) [2023-10-13 00:04:50,835][44959] Updated weights for policy 1, policy_version 96050 (0.0008) [2023-10-13 00:04:51,207][44959] Updated weights for policy 1, policy_version 96060 (0.0009) [2023-10-13 00:04:51,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 196280320. Throughput: 0: 1634.1, 1: 1639.5. Samples: 49069058. Policy #0 lag: (min: 10.0, avg: 14.6, max: 42.0) [2023-10-13 00:04:51,443][43579] Avg episode reward: [(0, '289.610'), (1, '284.680')] [2023-10-13 00:04:54,845][44958] Updated weights for policy 0, policy_version 95620 (0.0007) [2023-10-13 00:04:55,212][44958] Updated weights for policy 0, policy_version 95630 (0.0009) [2023-10-13 00:04:55,260][44959] Updated weights for policy 1, policy_version 96070 (0.0008) [2023-10-13 00:04:55,577][44958] Updated weights for policy 0, policy_version 95640 (0.0008) [2023-10-13 00:04:55,631][44959] Updated weights for policy 1, policy_version 96080 (0.0009) [2023-10-13 00:04:56,012][44959] Updated weights for policy 1, policy_version 96090 (0.0009) [2023-10-13 00:04:56,443][43579] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 196345856. Throughput: 0: 1633.4, 1: 1648.2. Samples: 49088916. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:04:56,444][43579] Avg episode reward: [(0, '294.820'), (1, '282.360')] [2023-10-13 00:04:59,918][44958] Updated weights for policy 0, policy_version 95650 (0.0008) [2023-10-13 00:05:00,290][44958] Updated weights for policy 0, policy_version 95660 (0.0008) [2023-10-13 00:05:00,302][44959] Updated weights for policy 1, policy_version 96100 (0.0008) [2023-10-13 00:05:00,650][44958] Updated weights for policy 0, policy_version 95670 (0.0010) [2023-10-13 00:05:00,675][44959] Updated weights for policy 1, policy_version 96110 (0.0007) [2023-10-13 00:05:01,029][44958] Updated weights for policy 0, policy_version 95680 (0.0008) [2023-10-13 00:05:01,041][44959] Updated weights for policy 1, policy_version 96120 (0.0009) [2023-10-13 00:05:01,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 196411392. Throughput: 0: 1630.7, 1: 1639.1. Samples: 49107030. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:01,443][43579] Avg episode reward: [(0, '292.290'), (1, '285.000')] [2023-10-13 00:05:04,967][44958] Updated weights for policy 0, policy_version 95690 (0.0009) [2023-10-13 00:05:05,243][44959] Updated weights for policy 1, policy_version 96130 (0.0009) [2023-10-13 00:05:05,331][44958] Updated weights for policy 0, policy_version 95700 (0.0008) [2023-10-13 00:05:05,613][44959] Updated weights for policy 1, policy_version 96140 (0.0007) [2023-10-13 00:05:05,711][44958] Updated weights for policy 0, policy_version 95710 (0.0010) [2023-10-13 00:05:05,982][44959] Updated weights for policy 1, policy_version 96150 (0.0008) [2023-10-13 00:05:06,352][44959] Updated weights for policy 1, policy_version 96160 (0.0011) [2023-10-13 00:05:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 196476928. Throughput: 0: 1626.3, 1: 1640.0. Samples: 49117980. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:06,443][43579] Avg episode reward: [(0, '294.980'), (1, '283.840')] [2023-10-13 00:05:10,193][44958] Updated weights for policy 0, policy_version 95720 (0.0008) [2023-10-13 00:05:10,505][44959] Updated weights for policy 1, policy_version 96170 (0.0009) [2023-10-13 00:05:10,562][44958] Updated weights for policy 0, policy_version 95730 (0.0008) [2023-10-13 00:05:10,869][44959] Updated weights for policy 1, policy_version 96180 (0.0008) [2023-10-13 00:05:10,933][44958] Updated weights for policy 0, policy_version 95740 (0.0009) [2023-10-13 00:05:11,235][44959] Updated weights for policy 1, policy_version 96190 (0.0008) [2023-10-13 00:05:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 196542464. Throughput: 0: 1624.9, 1: 1640.6. Samples: 49137512. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:11,443][43579] Avg episode reward: [(0, '295.470'), (1, '286.100')] [2023-10-13 00:05:15,440][44958] Updated weights for policy 0, policy_version 95750 (0.0007) [2023-10-13 00:05:15,502][44959] Updated weights for policy 1, policy_version 96200 (0.0008) [2023-10-13 00:05:15,816][44958] Updated weights for policy 0, policy_version 95760 (0.0008) [2023-10-13 00:05:15,870][44959] Updated weights for policy 1, policy_version 96210 (0.0007) [2023-10-13 00:05:16,190][44958] Updated weights for policy 0, policy_version 95770 (0.0008) [2023-10-13 00:05:16,237][44959] Updated weights for policy 1, policy_version 96220 (0.0009) [2023-10-13 00:05:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 196608000. Throughput: 0: 1624.0, 1: 1640.0. Samples: 49155858. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:16,444][43579] Avg episode reward: [(0, '296.230'), (1, '284.070')] [2023-10-13 00:05:20,339][44959] Updated weights for policy 1, policy_version 96230 (0.0009) [2023-10-13 00:05:20,371][44958] Updated weights for policy 0, policy_version 95780 (0.0009) [2023-10-13 00:05:20,701][44959] Updated weights for policy 1, policy_version 96240 (0.0009) [2023-10-13 00:05:20,732][44958] Updated weights for policy 0, policy_version 95790 (0.0007) [2023-10-13 00:05:21,070][44959] Updated weights for policy 1, policy_version 96250 (0.0007) [2023-10-13 00:05:21,107][44958] Updated weights for policy 0, policy_version 95800 (0.0009) [2023-10-13 00:05:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 196673536. Throughput: 0: 1616.1, 1: 1638.4. Samples: 49166576. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:21,443][43579] Avg episode reward: [(0, '294.220'), (1, '285.960')] [2023-10-13 00:05:25,279][44958] Updated weights for policy 0, policy_version 95810 (0.0007) [2023-10-13 00:05:25,471][44959] Updated weights for policy 1, policy_version 96260 (0.0009) [2023-10-13 00:05:25,658][44958] Updated weights for policy 0, policy_version 95820 (0.0009) [2023-10-13 00:05:25,864][44959] Updated weights for policy 1, policy_version 96270 (0.0008) [2023-10-13 00:05:26,022][44958] Updated weights for policy 0, policy_version 95830 (0.0008) [2023-10-13 00:05:26,236][44959] Updated weights for policy 1, policy_version 96280 (0.0008) [2023-10-13 00:05:26,395][44958] Updated weights for policy 0, policy_version 95840 (0.0007) [2023-10-13 00:05:26,443][43579] Fps is (10 sec: 9830.3, 60 sec: 12561.0, 300 sec: 13107.2). Total num frames: 196706304. Throughput: 0: 1627.5, 1: 1639.7. Samples: 49186660. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:26,444][43579] Avg episode reward: [(0, '294.200'), (1, '284.050')] [2023-10-13 00:05:30,262][44959] Updated weights for policy 1, policy_version 96290 (0.0009) [2023-10-13 00:05:30,630][44959] Updated weights for policy 1, policy_version 96300 (0.0009) [2023-10-13 00:05:30,757][44958] Updated weights for policy 0, policy_version 95850 (0.0007) [2023-10-13 00:05:30,990][44959] Updated weights for policy 1, policy_version 96310 (0.0008) [2023-10-13 00:05:31,128][44958] Updated weights for policy 0, policy_version 95860 (0.0007) [2023-10-13 00:05:31,356][44959] Updated weights for policy 1, policy_version 96320 (0.0008) [2023-10-13 00:05:31,442][43579] Fps is (10 sec: 9830.4, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 196771840. Throughput: 0: 1617.5, 1: 1639.6. Samples: 49204806. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:31,443][43579] Avg episode reward: [(0, '296.970'), (1, '282.150')] [2023-10-13 00:05:31,497][44958] Updated weights for policy 0, policy_version 95870 (0.0010) [2023-10-13 00:05:35,346][44959] Updated weights for policy 1, policy_version 96330 (0.0009) [2023-10-13 00:05:35,713][44959] Updated weights for policy 1, policy_version 96340 (0.0009) [2023-10-13 00:05:35,729][44958] Updated weights for policy 0, policy_version 95880 (0.0008) [2023-10-13 00:05:36,078][44959] Updated weights for policy 1, policy_version 96350 (0.0008) [2023-10-13 00:05:36,099][44958] Updated weights for policy 0, policy_version 95890 (0.0007) [2023-10-13 00:05:36,442][43579] Fps is (10 sec: 13107.6, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 196837376. Throughput: 0: 1607.8, 1: 1642.3. Samples: 49215310. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:36,443][43579] Avg episode reward: [(0, '294.450'), (1, '282.390')] [2023-10-13 00:05:36,479][44958] Updated weights for policy 0, policy_version 95900 (0.0009) [2023-10-13 00:05:40,335][44959] Updated weights for policy 1, policy_version 96360 (0.0008) [2023-10-13 00:05:40,667][44958] Updated weights for policy 0, policy_version 95910 (0.0007) [2023-10-13 00:05:40,701][44959] Updated weights for policy 1, policy_version 96370 (0.0007) [2023-10-13 00:05:41,039][44958] Updated weights for policy 0, policy_version 95920 (0.0009) [2023-10-13 00:05:41,061][44959] Updated weights for policy 1, policy_version 96380 (0.0007) [2023-10-13 00:05:41,402][44958] Updated weights for policy 0, policy_version 95930 (0.0011) [2023-10-13 00:05:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 196902912. Throughput: 0: 1620.4, 1: 1632.7. Samples: 49235306. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:41,443][43579] Avg episode reward: [(0, '290.230'), (1, '287.920')] [2023-10-13 00:05:45,323][44959] Updated weights for policy 1, policy_version 96390 (0.0008) [2023-10-13 00:05:45,446][44958] Updated weights for policy 0, policy_version 95940 (0.0008) [2023-10-13 00:05:45,677][44959] Updated weights for policy 1, policy_version 96400 (0.0008) [2023-10-13 00:05:45,828][44958] Updated weights for policy 0, policy_version 95950 (0.0009) [2023-10-13 00:05:46,040][44959] Updated weights for policy 1, policy_version 96410 (0.0009) [2023-10-13 00:05:46,203][44958] Updated weights for policy 0, policy_version 95960 (0.0008) [2023-10-13 00:05:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 196968448. Throughput: 0: 1626.1, 1: 1635.3. Samples: 49253792. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:46,443][43579] Avg episode reward: [(0, '290.120'), (1, '285.010')] [2023-10-13 00:05:46,450][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000096416_98729984.pth... [2023-10-13 00:05:46,480][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000094880_97157120.pth [2023-10-13 00:05:46,497][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000095968_98271232.pth... [2023-10-13 00:05:46,535][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000094432_96698368.pth [2023-10-13 00:05:50,418][44958] Updated weights for policy 0, policy_version 95970 (0.0009) [2023-10-13 00:05:50,435][44959] Updated weights for policy 1, policy_version 96420 (0.0007) [2023-10-13 00:05:50,787][44958] Updated weights for policy 0, policy_version 95980 (0.0007) [2023-10-13 00:05:50,815][44959] Updated weights for policy 1, policy_version 96430 (0.0007) [2023-10-13 00:05:51,160][44958] Updated weights for policy 0, policy_version 95990 (0.0008) [2023-10-13 00:05:51,179][44959] Updated weights for policy 1, policy_version 96440 (0.0008) [2023-10-13 00:05:51,442][43579] Fps is (10 sec: 9830.3, 60 sec: 12014.9, 300 sec: 12996.1). Total num frames: 197001216. Throughput: 0: 1613.5, 1: 1635.5. Samples: 49264184. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:51,443][43579] Avg episode reward: [(0, '289.420'), (1, '281.820')] [2023-10-13 00:05:51,526][44958] Updated weights for policy 0, policy_version 96000 (0.0009) [2023-10-13 00:05:55,223][44959] Updated weights for policy 1, policy_version 96450 (0.0010) [2023-10-13 00:05:55,595][44959] Updated weights for policy 1, policy_version 96460 (0.0009) [2023-10-13 00:05:55,801][44958] Updated weights for policy 0, policy_version 96010 (0.0009) [2023-10-13 00:05:55,966][44959] Updated weights for policy 1, policy_version 96470 (0.0007) [2023-10-13 00:05:56,178][44958] Updated weights for policy 0, policy_version 96020 (0.0009) [2023-10-13 00:05:56,325][44959] Updated weights for policy 1, policy_version 96480 (0.0009) [2023-10-13 00:05:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 197099520. Throughput: 0: 1624.4, 1: 1634.4. Samples: 49284156. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-13 00:05:56,443][43579] Avg episode reward: [(0, '289.660'), (1, '287.720')] [2023-10-13 00:05:56,541][44958] Updated weights for policy 0, policy_version 96030 (0.0009) [2023-10-13 00:06:00,583][44959] Updated weights for policy 1, policy_version 96490 (0.0007) [2023-10-13 00:06:00,771][44958] Updated weights for policy 0, policy_version 96040 (0.0008) [2023-10-13 00:06:00,956][44959] Updated weights for policy 1, policy_version 96500 (0.0007) [2023-10-13 00:06:01,132][44958] Updated weights for policy 0, policy_version 96050 (0.0007) [2023-10-13 00:06:01,312][44959] Updated weights for policy 1, policy_version 96510 (0.0009) [2023-10-13 00:06:01,442][43579] Fps is (10 sec: 16384.1, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 197165056. Throughput: 0: 1628.3, 1: 1639.4. Samples: 49302904. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:01,443][43579] Avg episode reward: [(0, '285.180'), (1, '284.700')] [2023-10-13 00:06:01,503][44958] Updated weights for policy 0, policy_version 96060 (0.0008) [2023-10-13 00:06:05,321][44959] Updated weights for policy 1, policy_version 96520 (0.0009) [2023-10-13 00:06:05,692][44959] Updated weights for policy 1, policy_version 96530 (0.0009) [2023-10-13 00:06:05,712][44958] Updated weights for policy 0, policy_version 96070 (0.0009) [2023-10-13 00:06:06,068][44959] Updated weights for policy 1, policy_version 96540 (0.0007) [2023-10-13 00:06:06,083][44958] Updated weights for policy 0, policy_version 96080 (0.0010) [2023-10-13 00:06:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 197230592. Throughput: 0: 1619.9, 1: 1640.0. Samples: 49313274. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:06,443][43579] Avg episode reward: [(0, '288.670'), (1, '283.520')] [2023-10-13 00:06:06,458][44958] Updated weights for policy 0, policy_version 96090 (0.0009) [2023-10-13 00:06:10,364][44959] Updated weights for policy 1, policy_version 96550 (0.0009) [2023-10-13 00:06:10,751][44959] Updated weights for policy 1, policy_version 96560 (0.0008) [2023-10-13 00:06:10,872][44958] Updated weights for policy 0, policy_version 96100 (0.0008) [2023-10-13 00:06:11,115][44959] Updated weights for policy 1, policy_version 96570 (0.0007) [2023-10-13 00:06:11,251][44958] Updated weights for policy 0, policy_version 96110 (0.0008) [2023-10-13 00:06:11,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 197296128. Throughput: 0: 1616.8, 1: 1638.1. Samples: 49333126. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:11,443][43579] Avg episode reward: [(0, '290.310'), (1, '283.270')] [2023-10-13 00:06:11,615][44958] Updated weights for policy 0, policy_version 96120 (0.0010) [2023-10-13 00:06:15,158][44959] Updated weights for policy 1, policy_version 96580 (0.0009) [2023-10-13 00:06:15,537][44959] Updated weights for policy 1, policy_version 96590 (0.0009) [2023-10-13 00:06:15,813][44958] Updated weights for policy 0, policy_version 96130 (0.0009) [2023-10-13 00:06:15,899][44959] Updated weights for policy 1, policy_version 96600 (0.0009) [2023-10-13 00:06:16,184][44958] Updated weights for policy 0, policy_version 96140 (0.0008) [2023-10-13 00:06:16,442][43579] Fps is (10 sec: 13107.3, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 197361664. Throughput: 0: 1631.2, 1: 1636.2. Samples: 49351842. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:16,443][43579] Avg episode reward: [(0, '290.060'), (1, '283.720')] [2023-10-13 00:06:16,554][44958] Updated weights for policy 0, policy_version 96150 (0.0009) [2023-10-13 00:06:16,926][44958] Updated weights for policy 0, policy_version 96160 (0.0010) [2023-10-13 00:06:19,990][44959] Updated weights for policy 1, policy_version 96610 (0.0009) [2023-10-13 00:06:20,368][44959] Updated weights for policy 1, policy_version 96620 (0.0007) [2023-10-13 00:06:20,735][44959] Updated weights for policy 1, policy_version 96630 (0.0008) [2023-10-13 00:06:21,050][44958] Updated weights for policy 0, policy_version 96170 (0.0007) [2023-10-13 00:06:21,103][44959] Updated weights for policy 1, policy_version 96640 (0.0009) [2023-10-13 00:06:21,426][44958] Updated weights for policy 0, policy_version 96180 (0.0007) [2023-10-13 00:06:21,442][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 13107.2). Total num frames: 197427200. Throughput: 0: 1621.2, 1: 1642.4. Samples: 49362170. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:21,443][43579] Avg episode reward: [(0, '289.270'), (1, '280.570')] [2023-10-13 00:06:21,792][44958] Updated weights for policy 0, policy_version 96190 (0.0007) [2023-10-13 00:06:25,263][44959] Updated weights for policy 1, policy_version 96650 (0.0007) [2023-10-13 00:06:25,632][44959] Updated weights for policy 1, policy_version 96660 (0.0007) [2023-10-13 00:06:25,937][44958] Updated weights for policy 0, policy_version 96200 (0.0008) [2023-10-13 00:06:25,991][44959] Updated weights for policy 1, policy_version 96670 (0.0007) [2023-10-13 00:06:26,309][44958] Updated weights for policy 0, policy_version 96210 (0.0008) [2023-10-13 00:06:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 197492736. Throughput: 0: 1624.5, 1: 1642.1. Samples: 49382302. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:26,443][43579] Avg episode reward: [(0, '290.560'), (1, '279.430')] [2023-10-13 00:06:26,682][44958] Updated weights for policy 0, policy_version 96220 (0.0009) [2023-10-13 00:06:30,098][44959] Updated weights for policy 1, policy_version 96680 (0.0007) [2023-10-13 00:06:30,468][44959] Updated weights for policy 1, policy_version 96690 (0.0007) [2023-10-13 00:06:30,838][44959] Updated weights for policy 1, policy_version 96700 (0.0007) [2023-10-13 00:06:30,880][44958] Updated weights for policy 0, policy_version 96230 (0.0007) [2023-10-13 00:06:31,250][44958] Updated weights for policy 0, policy_version 96240 (0.0009) [2023-10-13 00:06:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 197558272. Throughput: 0: 1630.5, 1: 1643.1. Samples: 49401104. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:31,443][43579] Avg episode reward: [(0, '291.450'), (1, '282.910')] [2023-10-13 00:06:31,625][44958] Updated weights for policy 0, policy_version 96250 (0.0009) [2023-10-13 00:06:35,194][44959] Updated weights for policy 1, policy_version 96710 (0.0009) [2023-10-13 00:06:35,553][44959] Updated weights for policy 1, policy_version 96720 (0.0007) [2023-10-13 00:06:35,740][44958] Updated weights for policy 0, policy_version 96260 (0.0008) [2023-10-13 00:06:35,929][44959] Updated weights for policy 1, policy_version 96730 (0.0007) [2023-10-13 00:06:36,111][44958] Updated weights for policy 0, policy_version 96270 (0.0008) [2023-10-13 00:06:36,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 197623808. Throughput: 0: 1625.6, 1: 1651.4. Samples: 49411650. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:36,443][43579] Avg episode reward: [(0, '288.240'), (1, '282.950')] [2023-10-13 00:06:36,483][44958] Updated weights for policy 0, policy_version 96280 (0.0009) [2023-10-13 00:06:39,999][44959] Updated weights for policy 1, policy_version 96740 (0.0007) [2023-10-13 00:06:40,364][44959] Updated weights for policy 1, policy_version 96750 (0.0008) [2023-10-13 00:06:40,558][44958] Updated weights for policy 0, policy_version 96290 (0.0007) [2023-10-13 00:06:40,731][44959] Updated weights for policy 1, policy_version 96760 (0.0010) [2023-10-13 00:06:40,928][44958] Updated weights for policy 0, policy_version 96300 (0.0010) [2023-10-13 00:06:41,301][44958] Updated weights for policy 0, policy_version 96310 (0.0011) [2023-10-13 00:06:41,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 197689344. Throughput: 0: 1634.9, 1: 1646.7. Samples: 49431828. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:41,444][43579] Avg episode reward: [(0, '287.990'), (1, '283.560')] [2023-10-13 00:06:41,672][44958] Updated weights for policy 0, policy_version 96320 (0.0007) [2023-10-13 00:06:44,949][44959] Updated weights for policy 1, policy_version 96770 (0.0008) [2023-10-13 00:06:45,316][44959] Updated weights for policy 1, policy_version 96780 (0.0007) [2023-10-13 00:06:45,685][44959] Updated weights for policy 1, policy_version 96790 (0.0007) [2023-10-13 00:06:46,015][44958] Updated weights for policy 0, policy_version 96330 (0.0007) [2023-10-13 00:06:46,055][44959] Updated weights for policy 1, policy_version 96800 (0.0007) [2023-10-13 00:06:46,401][44958] Updated weights for policy 0, policy_version 96340 (0.0007) [2023-10-13 00:06:46,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 197754880. Throughput: 0: 1632.0, 1: 1647.5. Samples: 49450484. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:46,443][43579] Avg episode reward: [(0, '285.760'), (1, '289.870')] [2023-10-13 00:06:46,774][44958] Updated weights for policy 0, policy_version 96350 (0.0008) [2023-10-13 00:06:50,332][44959] Updated weights for policy 1, policy_version 96810 (0.0011) [2023-10-13 00:06:50,698][44959] Updated weights for policy 1, policy_version 96820 (0.0008) [2023-10-13 00:06:50,977][44958] Updated weights for policy 0, policy_version 96360 (0.0008) [2023-10-13 00:06:51,053][44959] Updated weights for policy 1, policy_version 96830 (0.0009) [2023-10-13 00:06:51,356][44958] Updated weights for policy 0, policy_version 96370 (0.0009) [2023-10-13 00:06:51,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 197820416. Throughput: 0: 1627.5, 1: 1649.1. Samples: 49460720. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:51,443][43579] Avg episode reward: [(0, '285.320'), (1, '291.440')] [2023-10-13 00:06:51,730][44958] Updated weights for policy 0, policy_version 96380 (0.0009) [2023-10-13 00:06:55,367][44959] Updated weights for policy 1, policy_version 96840 (0.0007) [2023-10-13 00:06:55,730][44959] Updated weights for policy 1, policy_version 96850 (0.0007) [2023-10-13 00:06:55,910][44958] Updated weights for policy 0, policy_version 96390 (0.0008) [2023-10-13 00:06:56,101][44959] Updated weights for policy 1, policy_version 96860 (0.0007) [2023-10-13 00:06:56,278][44958] Updated weights for policy 0, policy_version 96400 (0.0007) [2023-10-13 00:06:56,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 197885952. Throughput: 0: 1640.4, 1: 1643.6. Samples: 49480908. Policy #0 lag: (min: 15.0, avg: 17.8, max: 47.0) [2023-10-13 00:06:56,443][43579] Avg episode reward: [(0, '285.650'), (1, '289.400')] [2023-10-13 00:06:56,662][44958] Updated weights for policy 0, policy_version 96410 (0.0007) [2023-10-13 00:07:00,320][44959] Updated weights for policy 1, policy_version 96870 (0.0010) [2023-10-13 00:07:00,687][44959] Updated weights for policy 1, policy_version 96880 (0.0009) [2023-10-13 00:07:00,748][44958] Updated weights for policy 0, policy_version 96420 (0.0008) [2023-10-13 00:07:01,053][44959] Updated weights for policy 1, policy_version 96890 (0.0009) [2023-10-13 00:07:01,118][44958] Updated weights for policy 0, policy_version 96430 (0.0009) [2023-10-13 00:07:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 197951488. Throughput: 0: 1638.5, 1: 1646.8. Samples: 49499678. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:01,443][43579] Avg episode reward: [(0, '287.230'), (1, '284.090')] [2023-10-13 00:07:01,490][44958] Updated weights for policy 0, policy_version 96440 (0.0011) [2023-10-13 00:07:05,232][44959] Updated weights for policy 1, policy_version 96900 (0.0007) [2023-10-13 00:07:05,597][44959] Updated weights for policy 1, policy_version 96910 (0.0008) [2023-10-13 00:07:05,632][44958] Updated weights for policy 0, policy_version 96450 (0.0010) [2023-10-13 00:07:05,978][44959] Updated weights for policy 1, policy_version 96920 (0.0009) [2023-10-13 00:07:06,012][44958] Updated weights for policy 0, policy_version 96460 (0.0007) [2023-10-13 00:07:06,382][44958] Updated weights for policy 0, policy_version 96470 (0.0007) [2023-10-13 00:07:06,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 198017024. Throughput: 0: 1643.4, 1: 1638.0. Samples: 49509834. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:06,443][43579] Avg episode reward: [(0, '284.560'), (1, '286.980')] [2023-10-13 00:07:06,751][44958] Updated weights for policy 0, policy_version 96480 (0.0007) [2023-10-13 00:07:10,082][44959] Updated weights for policy 1, policy_version 96930 (0.0007) [2023-10-13 00:07:10,445][44959] Updated weights for policy 1, policy_version 96940 (0.0008) [2023-10-13 00:07:10,816][44959] Updated weights for policy 1, policy_version 96950 (0.0009) [2023-10-13 00:07:10,817][44958] Updated weights for policy 0, policy_version 96490 (0.0009) [2023-10-13 00:07:11,186][44959] Updated weights for policy 1, policy_version 96960 (0.0009) [2023-10-13 00:07:11,188][44958] Updated weights for policy 0, policy_version 96500 (0.0009) [2023-10-13 00:07:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 198082560. Throughput: 0: 1644.8, 1: 1642.1. Samples: 49530210. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:11,443][43579] Avg episode reward: [(0, '283.410'), (1, '286.980')] [2023-10-13 00:07:11,555][44958] Updated weights for policy 0, policy_version 96510 (0.0010) [2023-10-13 00:07:15,129][44959] Updated weights for policy 1, policy_version 96970 (0.0007) [2023-10-13 00:07:15,504][44959] Updated weights for policy 1, policy_version 96980 (0.0007) [2023-10-13 00:07:15,877][44959] Updated weights for policy 1, policy_version 96990 (0.0008) [2023-10-13 00:07:15,910][44958] Updated weights for policy 0, policy_version 96520 (0.0009) [2023-10-13 00:07:16,287][44958] Updated weights for policy 0, policy_version 96530 (0.0008) [2023-10-13 00:07:16,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 198148096. Throughput: 0: 1638.3, 1: 1639.9. Samples: 49548620. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:16,444][43579] Avg episode reward: [(0, '287.570'), (1, '286.390')] [2023-10-13 00:07:16,660][44958] Updated weights for policy 0, policy_version 96540 (0.0008) [2023-10-13 00:07:20,085][44959] Updated weights for policy 1, policy_version 97000 (0.0009) [2023-10-13 00:07:20,456][44959] Updated weights for policy 1, policy_version 97010 (0.0007) [2023-10-13 00:07:20,817][44959] Updated weights for policy 1, policy_version 97020 (0.0007) [2023-10-13 00:07:21,069][44958] Updated weights for policy 0, policy_version 96550 (0.0009) [2023-10-13 00:07:21,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 198213632. Throughput: 0: 1638.1, 1: 1639.0. Samples: 49559120. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:21,443][43579] Avg episode reward: [(0, '284.900'), (1, '288.450')] [2023-10-13 00:07:21,449][44958] Updated weights for policy 0, policy_version 96560 (0.0010) [2023-10-13 00:07:21,808][44958] Updated weights for policy 0, policy_version 96570 (0.0011) [2023-10-13 00:07:25,095][44959] Updated weights for policy 1, policy_version 97030 (0.0009) [2023-10-13 00:07:25,470][44959] Updated weights for policy 1, policy_version 97040 (0.0010) [2023-10-13 00:07:25,786][44958] Updated weights for policy 0, policy_version 96580 (0.0009) [2023-10-13 00:07:25,836][44959] Updated weights for policy 1, policy_version 97050 (0.0007) [2023-10-13 00:07:26,163][44958] Updated weights for policy 0, policy_version 96590 (0.0007) [2023-10-13 00:07:26,442][43579] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 198279168. Throughput: 0: 1637.3, 1: 1640.4. Samples: 49579322. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:26,443][43579] Avg episode reward: [(0, '284.670'), (1, '289.840')] [2023-10-13 00:07:26,536][44958] Updated weights for policy 0, policy_version 96600 (0.0009) [2023-10-13 00:07:29,967][44959] Updated weights for policy 1, policy_version 97060 (0.0007) [2023-10-13 00:07:30,329][44959] Updated weights for policy 1, policy_version 97070 (0.0008) [2023-10-13 00:07:30,694][44959] Updated weights for policy 1, policy_version 97080 (0.0009) [2023-10-13 00:07:30,736][44958] Updated weights for policy 0, policy_version 96610 (0.0010) [2023-10-13 00:07:31,137][44958] Updated weights for policy 0, policy_version 96620 (0.0009) [2023-10-13 00:07:31,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 198344704. Throughput: 0: 1646.0, 1: 1635.3. Samples: 49598142. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:31,443][43579] Avg episode reward: [(0, '285.610'), (1, '292.580')] [2023-10-13 00:07:31,504][44958] Updated weights for policy 0, policy_version 96630 (0.0008) [2023-10-13 00:07:31,868][44958] Updated weights for policy 0, policy_version 96640 (0.0009) [2023-10-13 00:07:34,897][44959] Updated weights for policy 1, policy_version 97090 (0.0008) [2023-10-13 00:07:35,274][44959] Updated weights for policy 1, policy_version 97100 (0.0007) [2023-10-13 00:07:35,634][44959] Updated weights for policy 1, policy_version 97110 (0.0007) [2023-10-13 00:07:36,004][44959] Updated weights for policy 1, policy_version 97120 (0.0009) [2023-10-13 00:07:36,046][44958] Updated weights for policy 0, policy_version 96650 (0.0008) [2023-10-13 00:07:36,418][44958] Updated weights for policy 0, policy_version 96660 (0.0008) [2023-10-13 00:07:36,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 198410240. Throughput: 0: 1647.2, 1: 1640.2. Samples: 49608650. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:36,444][43579] Avg episode reward: [(0, '293.810'), (1, '290.160')] [2023-10-13 00:07:36,798][44958] Updated weights for policy 0, policy_version 96670 (0.0007) [2023-10-13 00:07:40,268][44959] Updated weights for policy 1, policy_version 97130 (0.0009) [2023-10-13 00:07:40,640][44959] Updated weights for policy 1, policy_version 97140 (0.0008) [2023-10-13 00:07:41,011][44959] Updated weights for policy 1, policy_version 97150 (0.0008) [2023-10-13 00:07:41,024][44958] Updated weights for policy 0, policy_version 96680 (0.0007) [2023-10-13 00:07:41,392][44958] Updated weights for policy 0, policy_version 96690 (0.0008) [2023-10-13 00:07:41,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 198475776. Throughput: 0: 1639.2, 1: 1645.6. Samples: 49628724. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:41,443][43579] Avg episode reward: [(0, '295.270'), (1, '289.790')] [2023-10-13 00:07:41,766][44958] Updated weights for policy 0, policy_version 96700 (0.0008) [2023-10-13 00:07:44,967][44959] Updated weights for policy 1, policy_version 97160 (0.0007) [2023-10-13 00:07:45,340][44959] Updated weights for policy 1, policy_version 97170 (0.0008) [2023-10-13 00:07:45,703][44959] Updated weights for policy 1, policy_version 97180 (0.0009) [2023-10-13 00:07:45,800][44958] Updated weights for policy 0, policy_version 96710 (0.0007) [2023-10-13 00:07:46,166][44958] Updated weights for policy 0, policy_version 96720 (0.0007) [2023-10-13 00:07:46,443][43579] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 198541312. Throughput: 0: 1638.3, 1: 1643.6. Samples: 49647364. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:46,444][43579] Avg episode reward: [(0, '294.730'), (1, '287.220')] [2023-10-13 00:07:46,454][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000097184_99516416.pth... [2023-10-13 00:07:46,495][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000095648_97943552.pth [2023-10-13 00:07:46,537][44958] Updated weights for policy 0, policy_version 96730 (0.0008) [2023-10-13 00:07:46,760][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000096736_99057664.pth... [2023-10-13 00:07:46,798][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000095200_97484800.pth [2023-10-13 00:07:49,945][44959] Updated weights for policy 1, policy_version 97190 (0.0009) [2023-10-13 00:07:50,314][44959] Updated weights for policy 1, policy_version 97200 (0.0009) [2023-10-13 00:07:50,642][44958] Updated weights for policy 0, policy_version 96740 (0.0008) [2023-10-13 00:07:50,685][44959] Updated weights for policy 1, policy_version 97210 (0.0007) [2023-10-13 00:07:51,011][44958] Updated weights for policy 0, policy_version 96750 (0.0009) [2023-10-13 00:07:51,390][44958] Updated weights for policy 0, policy_version 96760 (0.0010) [2023-10-13 00:07:51,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 198606848. Throughput: 0: 1640.9, 1: 1650.3. Samples: 49657940. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:51,443][43579] Avg episode reward: [(0, '290.340'), (1, '286.980')] [2023-10-13 00:07:54,705][44959] Updated weights for policy 1, policy_version 97220 (0.0007) [2023-10-13 00:07:55,075][44959] Updated weights for policy 1, policy_version 97230 (0.0010) [2023-10-13 00:07:55,443][44959] Updated weights for policy 1, policy_version 97240 (0.0009) [2023-10-13 00:07:55,661][44958] Updated weights for policy 0, policy_version 96770 (0.0010) [2023-10-13 00:07:56,032][44958] Updated weights for policy 0, policy_version 96780 (0.0008) [2023-10-13 00:07:56,398][44958] Updated weights for policy 0, policy_version 96790 (0.0010) [2023-10-13 00:07:56,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 198672384. Throughput: 0: 1641.8, 1: 1642.9. Samples: 49678022. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-13 00:07:56,443][43579] Avg episode reward: [(0, '288.660'), (1, '285.880')] [2023-10-13 00:07:56,769][44958] Updated weights for policy 0, policy_version 96800 (0.0009) [2023-10-13 00:07:59,690][44959] Updated weights for policy 1, policy_version 97250 (0.0009) [2023-10-13 00:08:00,043][44959] Updated weights for policy 1, policy_version 97260 (0.0007) [2023-10-13 00:08:00,418][44959] Updated weights for policy 1, policy_version 97270 (0.0008) [2023-10-13 00:08:00,756][44958] Updated weights for policy 0, policy_version 96810 (0.0007) [2023-10-13 00:08:00,781][44959] Updated weights for policy 1, policy_version 97280 (0.0008) [2023-10-13 00:08:01,128][44958] Updated weights for policy 0, policy_version 96820 (0.0007) [2023-10-13 00:08:01,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 198737920. Throughput: 0: 1644.5, 1: 1649.8. Samples: 49696860. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:01,443][43579] Avg episode reward: [(0, '287.800'), (1, '285.240')] [2023-10-13 00:08:01,495][44958] Updated weights for policy 0, policy_version 96830 (0.0008) [2023-10-13 00:08:04,895][44959] Updated weights for policy 1, policy_version 97290 (0.0010) [2023-10-13 00:08:05,267][44959] Updated weights for policy 1, policy_version 97300 (0.0009) [2023-10-13 00:08:05,635][44959] Updated weights for policy 1, policy_version 97310 (0.0010) [2023-10-13 00:08:05,699][44958] Updated weights for policy 0, policy_version 96840 (0.0007) [2023-10-13 00:08:06,067][44958] Updated weights for policy 0, policy_version 96850 (0.0008) [2023-10-13 00:08:06,441][44958] Updated weights for policy 0, policy_version 96860 (0.0008) [2023-10-13 00:08:06,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 198803456. Throughput: 0: 1648.8, 1: 1651.5. Samples: 49707634. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:06,443][43579] Avg episode reward: [(0, '286.470'), (1, '281.890')] [2023-10-13 00:08:09,675][44959] Updated weights for policy 1, policy_version 97320 (0.0010) [2023-10-13 00:08:10,055][44959] Updated weights for policy 1, policy_version 97330 (0.0008) [2023-10-13 00:08:10,422][44959] Updated weights for policy 1, policy_version 97340 (0.0007) [2023-10-13 00:08:10,794][44958] Updated weights for policy 0, policy_version 96870 (0.0009) [2023-10-13 00:08:11,170][44958] Updated weights for policy 0, policy_version 96880 (0.0009) [2023-10-13 00:08:11,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 198868992. Throughput: 0: 1640.4, 1: 1643.2. Samples: 49727086. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:11,444][43579] Avg episode reward: [(0, '285.490'), (1, '282.650')] [2023-10-13 00:08:11,541][44958] Updated weights for policy 0, policy_version 96890 (0.0007) [2023-10-13 00:08:14,591][44959] Updated weights for policy 1, policy_version 97350 (0.0009) [2023-10-13 00:08:14,955][44959] Updated weights for policy 1, policy_version 97360 (0.0008) [2023-10-13 00:08:15,321][44959] Updated weights for policy 1, policy_version 97370 (0.0008) [2023-10-13 00:08:15,889][44958] Updated weights for policy 0, policy_version 96900 (0.0008) [2023-10-13 00:08:16,278][44958] Updated weights for policy 0, policy_version 96910 (0.0009) [2023-10-13 00:08:16,443][43579] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 198934528. Throughput: 0: 1638.4, 1: 1655.8. Samples: 49746382. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:16,444][43579] Avg episode reward: [(0, '281.590'), (1, '280.770')] [2023-10-13 00:08:16,646][44958] Updated weights for policy 0, policy_version 96920 (0.0008) [2023-10-13 00:08:19,351][44959] Updated weights for policy 1, policy_version 97380 (0.0007) [2023-10-13 00:08:19,706][44959] Updated weights for policy 1, policy_version 97390 (0.0010) [2023-10-13 00:08:20,079][44959] Updated weights for policy 1, policy_version 97400 (0.0011) [2023-10-13 00:08:20,713][44958] Updated weights for policy 0, policy_version 96930 (0.0008) [2023-10-13 00:08:21,085][44958] Updated weights for policy 0, policy_version 96940 (0.0008) [2023-10-13 00:08:21,442][43579] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199000064. Throughput: 0: 1642.2, 1: 1657.6. Samples: 49757138. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:21,443][43579] Avg episode reward: [(0, '286.550'), (1, '278.370')] [2023-10-13 00:08:21,453][44958] Updated weights for policy 0, policy_version 96950 (0.0011) [2023-10-13 00:08:21,829][44958] Updated weights for policy 0, policy_version 96960 (0.0010) [2023-10-13 00:08:24,518][44959] Updated weights for policy 1, policy_version 97410 (0.0009) [2023-10-13 00:08:24,923][44959] Updated weights for policy 1, policy_version 97420 (0.0008) [2023-10-13 00:08:25,284][44959] Updated weights for policy 1, policy_version 97430 (0.0010) [2023-10-13 00:08:25,651][44959] Updated weights for policy 1, policy_version 97440 (0.0010) [2023-10-13 00:08:25,989][44958] Updated weights for policy 0, policy_version 96970 (0.0007) [2023-10-13 00:08:26,361][44958] Updated weights for policy 0, policy_version 96980 (0.0008) [2023-10-13 00:08:26,443][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 12996.1). Total num frames: 199065600. Throughput: 0: 1645.7, 1: 1647.6. Samples: 49776926. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:26,444][43579] Avg episode reward: [(0, '289.100'), (1, '277.040')] [2023-10-13 00:08:26,739][44958] Updated weights for policy 0, policy_version 96990 (0.0009) [2023-10-13 00:08:29,496][44959] Updated weights for policy 1, policy_version 97450 (0.0009) [2023-10-13 00:08:29,865][44959] Updated weights for policy 1, policy_version 97460 (0.0008) [2023-10-13 00:08:30,231][44959] Updated weights for policy 1, policy_version 97470 (0.0010) [2023-10-13 00:08:30,879][44958] Updated weights for policy 0, policy_version 97000 (0.0010) [2023-10-13 00:08:31,263][44958] Updated weights for policy 0, policy_version 97010 (0.0008) [2023-10-13 00:08:31,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199131136. Throughput: 0: 1644.1, 1: 1657.8. Samples: 49795948. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:31,443][43579] Avg episode reward: [(0, '289.040'), (1, '282.910')] [2023-10-13 00:08:31,643][44958] Updated weights for policy 0, policy_version 97020 (0.0009) [2023-10-13 00:08:34,340][44959] Updated weights for policy 1, policy_version 97480 (0.0008) [2023-10-13 00:08:34,700][44959] Updated weights for policy 1, policy_version 97490 (0.0010) [2023-10-13 00:08:35,077][44959] Updated weights for policy 1, policy_version 97500 (0.0009) [2023-10-13 00:08:35,759][44958] Updated weights for policy 0, policy_version 97030 (0.0009) [2023-10-13 00:08:36,132][44958] Updated weights for policy 0, policy_version 97040 (0.0009) [2023-10-13 00:08:36,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 199196672. Throughput: 0: 1641.4, 1: 1662.2. Samples: 49806602. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:36,443][43579] Avg episode reward: [(0, '288.480'), (1, '287.810')] [2023-10-13 00:08:36,510][44958] Updated weights for policy 0, policy_version 97050 (0.0010) [2023-10-13 00:08:39,176][44959] Updated weights for policy 1, policy_version 97510 (0.0008) [2023-10-13 00:08:39,536][44959] Updated weights for policy 1, policy_version 97520 (0.0008) [2023-10-13 00:08:39,907][44959] Updated weights for policy 1, policy_version 97530 (0.0009) [2023-10-13 00:08:40,542][44958] Updated weights for policy 0, policy_version 97060 (0.0010) [2023-10-13 00:08:40,920][44958] Updated weights for policy 0, policy_version 97070 (0.0009) [2023-10-13 00:08:41,287][44958] Updated weights for policy 0, policy_version 97080 (0.0009) [2023-10-13 00:08:41,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199262208. Throughput: 0: 1644.4, 1: 1647.5. Samples: 49826158. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:41,443][43579] Avg episode reward: [(0, '286.070'), (1, '289.020')] [2023-10-13 00:08:44,092][44959] Updated weights for policy 1, policy_version 97540 (0.0009) [2023-10-13 00:08:44,467][44959] Updated weights for policy 1, policy_version 97550 (0.0007) [2023-10-13 00:08:44,830][44959] Updated weights for policy 1, policy_version 97560 (0.0008) [2023-10-13 00:08:45,391][44958] Updated weights for policy 0, policy_version 97090 (0.0008) [2023-10-13 00:08:45,761][44958] Updated weights for policy 0, policy_version 97100 (0.0010) [2023-10-13 00:08:46,137][44958] Updated weights for policy 0, policy_version 97110 (0.0008) [2023-10-13 00:08:46,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 199327744. Throughput: 0: 1638.5, 1: 1659.0. Samples: 49845248. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:46,443][43579] Avg episode reward: [(0, '289.870'), (1, '285.070')] [2023-10-13 00:08:46,511][44958] Updated weights for policy 0, policy_version 97120 (0.0009) [2023-10-13 00:08:49,054][44959] Updated weights for policy 1, policy_version 97570 (0.0007) [2023-10-13 00:08:49,422][44959] Updated weights for policy 1, policy_version 97580 (0.0007) [2023-10-13 00:08:49,791][44959] Updated weights for policy 1, policy_version 97590 (0.0009) [2023-10-13 00:08:50,159][44959] Updated weights for policy 1, policy_version 97600 (0.0008) [2023-10-13 00:08:50,638][44958] Updated weights for policy 0, policy_version 97130 (0.0008) [2023-10-13 00:08:50,997][44958] Updated weights for policy 0, policy_version 97140 (0.0009) [2023-10-13 00:08:51,367][44958] Updated weights for policy 0, policy_version 97150 (0.0008) [2023-10-13 00:08:51,442][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 199426048. Throughput: 0: 1641.6, 1: 1655.2. Samples: 49855994. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-13 00:08:51,443][43579] Avg episode reward: [(0, '289.800'), (1, '282.080')] [2023-10-13 00:08:54,211][44959] Updated weights for policy 1, policy_version 97610 (0.0008) [2023-10-13 00:08:54,582][44959] Updated weights for policy 1, policy_version 97620 (0.0007) [2023-10-13 00:08:54,954][44959] Updated weights for policy 1, policy_version 97630 (0.0007) [2023-10-13 00:08:55,459][44958] Updated weights for policy 0, policy_version 97160 (0.0009) [2023-10-13 00:08:55,830][44958] Updated weights for policy 0, policy_version 97170 (0.0010) [2023-10-13 00:08:56,203][44958] Updated weights for policy 0, policy_version 97180 (0.0010) [2023-10-13 00:08:56,443][43579] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13107.2). Total num frames: 199491584. Throughput: 0: 1648.7, 1: 1646.8. Samples: 49875380. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:08:56,443][43579] Avg episode reward: [(0, '289.170'), (1, '280.240')] [2023-10-13 00:08:59,232][44959] Updated weights for policy 1, policy_version 97640 (0.0007) [2023-10-13 00:08:59,612][44959] Updated weights for policy 1, policy_version 97650 (0.0008) [2023-10-13 00:08:59,974][44959] Updated weights for policy 1, policy_version 97660 (0.0010) [2023-10-13 00:09:00,654][44958] Updated weights for policy 0, policy_version 97190 (0.0007) [2023-10-13 00:09:01,032][44958] Updated weights for policy 0, policy_version 97200 (0.0008) [2023-10-13 00:09:01,409][44958] Updated weights for policy 0, policy_version 97210 (0.0008) [2023-10-13 00:09:01,443][43579] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199524352. Throughput: 0: 1636.3, 1: 1657.4. Samples: 49894600. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:01,443][43579] Avg episode reward: [(0, '289.640'), (1, '279.660')] [2023-10-13 00:09:04,125][44959] Updated weights for policy 1, policy_version 97670 (0.0008) [2023-10-13 00:09:04,503][44959] Updated weights for policy 1, policy_version 97680 (0.0007) [2023-10-13 00:09:04,869][44959] Updated weights for policy 1, policy_version 97690 (0.0007) [2023-10-13 00:09:05,645][44958] Updated weights for policy 0, policy_version 97220 (0.0008) [2023-10-13 00:09:06,007][44958] Updated weights for policy 0, policy_version 97230 (0.0007) [2023-10-13 00:09:06,380][44958] Updated weights for policy 0, policy_version 97240 (0.0007) [2023-10-13 00:09:06,443][43579] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199589888. Throughput: 0: 1637.1, 1: 1652.3. Samples: 49905164. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:06,444][43579] Avg episode reward: [(0, '289.080'), (1, '279.450')] [2023-10-13 00:09:08,996][44959] Updated weights for policy 1, policy_version 97700 (0.0007) [2023-10-13 00:09:09,358][44959] Updated weights for policy 1, policy_version 97710 (0.0009) [2023-10-13 00:09:09,735][44959] Updated weights for policy 1, policy_version 97720 (0.0009) [2023-10-13 00:09:10,574][44958] Updated weights for policy 0, policy_version 97250 (0.0009) [2023-10-13 00:09:10,941][44958] Updated weights for policy 0, policy_version 97260 (0.0007) [2023-10-13 00:09:11,310][44958] Updated weights for policy 0, policy_version 97270 (0.0008) [2023-10-13 00:09:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199655424. Throughput: 0: 1639.6, 1: 1639.3. Samples: 49924478. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:11,444][43579] Avg episode reward: [(0, '289.320'), (1, '279.230')] [2023-10-13 00:09:11,684][44958] Updated weights for policy 0, policy_version 97280 (0.0007) [2023-10-13 00:09:14,055][44959] Updated weights for policy 1, policy_version 97730 (0.0009) [2023-10-13 00:09:14,453][44959] Updated weights for policy 1, policy_version 97740 (0.0008) [2023-10-13 00:09:14,816][44959] Updated weights for policy 1, policy_version 97750 (0.0008) [2023-10-13 00:09:15,183][44959] Updated weights for policy 1, policy_version 97760 (0.0008) [2023-10-13 00:09:15,836][44958] Updated weights for policy 0, policy_version 97290 (0.0008) [2023-10-13 00:09:16,214][44958] Updated weights for policy 0, policy_version 97300 (0.0008) [2023-10-13 00:09:16,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 12996.1). Total num frames: 199720960. Throughput: 0: 1639.8, 1: 1647.6. Samples: 49943882. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:16,443][43579] Avg episode reward: [(0, '289.330'), (1, '286.850')] [2023-10-13 00:09:16,596][44958] Updated weights for policy 0, policy_version 97310 (0.0009) [2023-10-13 00:09:19,142][44959] Updated weights for policy 1, policy_version 97770 (0.0009) [2023-10-13 00:09:19,517][44959] Updated weights for policy 1, policy_version 97780 (0.0007) [2023-10-13 00:09:19,886][44959] Updated weights for policy 1, policy_version 97790 (0.0007) [2023-10-13 00:09:20,972][44958] Updated weights for policy 0, policy_version 97320 (0.0010) [2023-10-13 00:09:21,355][44958] Updated weights for policy 0, policy_version 97330 (0.0011) [2023-10-13 00:09:21,443][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199786496. Throughput: 0: 1641.0, 1: 1641.7. Samples: 49954326. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:21,444][43579] Avg episode reward: [(0, '283.750'), (1, '289.900')] [2023-10-13 00:09:21,724][44958] Updated weights for policy 0, policy_version 97340 (0.0010) [2023-10-13 00:09:24,007][44959] Updated weights for policy 1, policy_version 97800 (0.0007) [2023-10-13 00:09:24,387][44959] Updated weights for policy 1, policy_version 97810 (0.0009) [2023-10-13 00:09:24,745][44959] Updated weights for policy 1, policy_version 97820 (0.0009) [2023-10-13 00:09:25,906][44958] Updated weights for policy 0, policy_version 97350 (0.0009) [2023-10-13 00:09:26,272][44958] Updated weights for policy 0, policy_version 97360 (0.0008) [2023-10-13 00:09:26,442][43579] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199852032. Throughput: 0: 1635.2, 1: 1642.0. Samples: 49973634. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:26,443][43579] Avg episode reward: [(0, '285.670'), (1, '287.250')] [2023-10-13 00:09:26,643][44958] Updated weights for policy 0, policy_version 97370 (0.0010) [2023-10-13 00:09:28,964][44959] Updated weights for policy 1, policy_version 97830 (0.0010) [2023-10-13 00:09:29,331][44959] Updated weights for policy 1, policy_version 97840 (0.0007) [2023-10-13 00:09:29,694][44959] Updated weights for policy 1, policy_version 97850 (0.0007) [2023-10-13 00:09:30,670][44958] Updated weights for policy 0, policy_version 97380 (0.0009) [2023-10-13 00:09:31,034][44958] Updated weights for policy 0, policy_version 97390 (0.0009) [2023-10-13 00:09:31,403][44958] Updated weights for policy 0, policy_version 97400 (0.0009) [2023-10-13 00:09:31,442][43579] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199917568. Throughput: 0: 1637.6, 1: 1651.3. Samples: 49993248. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:31,443][43579] Avg episode reward: [(0, '282.190'), (1, '287.320')] [2023-10-13 00:09:34,048][44959] Updated weights for policy 1, policy_version 97860 (0.0008) [2023-10-13 00:09:34,423][44959] Updated weights for policy 1, policy_version 97870 (0.0007) [2023-10-13 00:09:34,793][44959] Updated weights for policy 1, policy_version 97880 (0.0008) [2023-10-13 00:09:35,501][44958] Updated weights for policy 0, policy_version 97410 (0.0010) [2023-10-13 00:09:35,865][44958] Updated weights for policy 0, policy_version 97420 (0.0010) [2023-10-13 00:09:36,249][44958] Updated weights for policy 0, policy_version 97430 (0.0007) [2023-10-13 00:09:36,442][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 199983104. Throughput: 0: 1633.2, 1: 1649.6. Samples: 50003724. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:36,443][43579] Avg episode reward: [(0, '275.080'), (1, '283.070')] [2023-10-13 00:09:36,620][44958] Updated weights for policy 0, policy_version 97440 (0.0007) [2023-10-13 00:09:39,161][44959] Updated weights for policy 1, policy_version 97890 (0.0009) [2023-10-13 00:09:39,525][44959] Updated weights for policy 1, policy_version 97900 (0.0010) [2023-10-13 00:09:39,886][44959] Updated weights for policy 1, policy_version 97910 (0.0009) [2023-10-13 00:09:40,248][44959] Updated weights for policy 1, policy_version 97920 (0.0007) [2023-10-13 00:09:40,744][44958] Updated weights for policy 0, policy_version 97450 (0.0009) [2023-10-13 00:09:41,118][44958] Updated weights for policy 0, policy_version 97460 (0.0007) [2023-10-13 00:09:41,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 200048640. Throughput: 0: 1636.5, 1: 1652.3. Samples: 50023374. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:41,443][43579] Avg episode reward: [(0, '273.390'), (1, '285.890')] [2023-10-13 00:09:41,497][44958] Updated weights for policy 0, policy_version 97470 (0.0008) [2023-10-13 00:09:44,229][44959] Updated weights for policy 1, policy_version 97930 (0.0009) [2023-10-13 00:09:44,599][44959] Updated weights for policy 1, policy_version 97940 (0.0008) [2023-10-13 00:09:44,968][44959] Updated weights for policy 1, policy_version 97950 (0.0009) [2023-10-13 00:09:45,800][44958] Updated weights for policy 0, policy_version 97480 (0.0008) [2023-10-13 00:09:46,163][44958] Updated weights for policy 0, policy_version 97490 (0.0008) [2023-10-13 00:09:46,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 200114176. Throughput: 0: 1642.5, 1: 1651.2. Samples: 50042818. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:46,443][43579] Avg episode reward: [(0, '275.240'), (1, '283.210')] [2023-10-13 00:09:46,451][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000097952_100302848.pth... [2023-10-13 00:09:46,480][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000096416_98729984.pth [2023-10-13 00:09:46,541][44958] Updated weights for policy 0, policy_version 97500 (0.0010) [2023-10-13 00:09:46,678][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000097504_99844096.pth... [2023-10-13 00:09:46,707][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000095968_98271232.pth [2023-10-13 00:09:49,033][44959] Updated weights for policy 1, policy_version 97960 (0.0008) [2023-10-13 00:09:49,405][44959] Updated weights for policy 1, policy_version 97970 (0.0009) [2023-10-13 00:09:49,783][44959] Updated weights for policy 1, policy_version 97980 (0.0010) [2023-10-13 00:09:50,727][44958] Updated weights for policy 0, policy_version 97510 (0.0008) [2023-10-13 00:09:51,092][44958] Updated weights for policy 0, policy_version 97520 (0.0010) [2023-10-13 00:09:51,443][43579] Fps is (10 sec: 13106.6, 60 sec: 12561.0, 300 sec: 12996.1). Total num frames: 200179712. Throughput: 0: 1641.2, 1: 1646.5. Samples: 50053112. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:51,444][43579] Avg episode reward: [(0, '277.320'), (1, '283.340')] [2023-10-13 00:09:51,474][44958] Updated weights for policy 0, policy_version 97530 (0.0007) [2023-10-13 00:09:53,697][44959] Updated weights for policy 1, policy_version 97990 (0.0010) [2023-10-13 00:09:54,058][44959] Updated weights for policy 1, policy_version 98000 (0.0007) [2023-10-13 00:09:54,427][44959] Updated weights for policy 1, policy_version 98010 (0.0007) [2023-10-13 00:09:55,631][44958] Updated weights for policy 0, policy_version 97540 (0.0009) [2023-10-13 00:09:56,007][44958] Updated weights for policy 0, policy_version 97550 (0.0011) [2023-10-13 00:09:56,383][44958] Updated weights for policy 0, policy_version 97560 (0.0008) [2023-10-13 00:09:56,443][43579] Fps is (10 sec: 13107.2, 60 sec: 12561.1, 300 sec: 12996.1). Total num frames: 200245248. Throughput: 0: 1639.7, 1: 1653.5. Samples: 50072672. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-13 00:09:56,443][43579] Avg episode reward: [(0, '273.290'), (1, '284.000')] [2023-10-13 00:09:58,463][44959] Updated weights for policy 1, policy_version 98020 (0.0008) [2023-10-13 00:09:58,860][44959] Updated weights for policy 1, policy_version 98030 (0.0010) [2023-10-13 00:09:59,228][44959] Updated weights for policy 1, policy_version 98040 (0.0010) [2023-10-13 00:10:00,591][44958] Updated weights for policy 0, policy_version 97570 (0.0008) [2023-10-13 00:10:00,960][44958] Updated weights for policy 0, policy_version 97580 (0.0011) [2023-10-13 00:10:01,318][44958] Updated weights for policy 0, policy_version 97590 (0.0009) [2023-10-13 00:10:01,442][43579] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 200310784. Throughput: 0: 1636.6, 1: 1657.7. Samples: 50092126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:10:01,443][43579] Avg episode reward: [(0, '273.570'), (1, '286.430')] [2023-10-13 00:10:01,697][44958] Updated weights for policy 0, policy_version 97600 (0.0009) [2023-10-13 00:10:03,353][44959] Updated weights for policy 1, policy_version 98050 (0.0009) [2023-10-13 00:10:03,723][44959] Updated weights for policy 1, policy_version 98060 (0.0009) [2023-10-13 00:10:04,098][44959] Updated weights for policy 1, policy_version 98070 (0.0007) [2023-10-13 00:10:04,462][44959] Updated weights for policy 1, policy_version 98080 (0.0008) [2023-10-13 00:10:05,782][44958] Updated weights for policy 0, policy_version 97610 (0.0008) [2023-10-13 00:10:06,149][44958] Updated weights for policy 0, policy_version 97620 (0.0007) [2023-10-13 00:10:06,442][43579] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 200376320. Throughput: 0: 1639.3, 1: 1644.1. Samples: 50102082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:10:06,443][43579] Avg episode reward: [(0, '280.070'), (1, '287.820')] [2023-10-13 00:10:06,510][44958] Updated weights for policy 0, policy_version 97630 (0.0010) [2023-10-13 00:10:08,665][44959] Updated weights for policy 1, policy_version 98090 (0.0007) [2023-10-13 00:10:09,038][44959] Updated weights for policy 1, policy_version 98100 (0.0008) [2023-10-13 00:10:09,397][44959] Updated weights for policy 1, policy_version 98110 (0.0009) [2023-10-13 00:10:10,854][44958] Updated weights for policy 0, policy_version 97640 (0.0008) [2023-10-13 00:10:11,213][44958] Updated weights for policy 0, policy_version 97650 (0.0008) [2023-10-13 00:10:11,443][43579] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12996.1). Total num frames: 200441856. Throughput: 0: 1641.9, 1: 1657.2. Samples: 50122092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-13 00:10:11,444][43579] Avg episode reward: [(0, '282.230'), (1, '288.160')] [2023-10-13 00:10:11,589][44958] Updated weights for policy 0, policy_version 97660 (0.0011) [2023-10-13 00:10:13,584][44959] Updated weights for policy 1, policy_version 98120 (0.0010) [2023-10-13 00:10:13,954][44959] Updated weights for policy 1, policy_version 98130 (0.0009) [2023-10-13 00:10:14,328][44959] Updated weights for policy 1, policy_version 98140 (0.0007) [2023-10-13 00:10:14,476][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000098144_100499456.pth... [2023-10-13 00:10:14,476][45006] Stopping RolloutWorker_w12... [2023-10-13 00:10:14,476][45006] Loop rollout_proc12_evt_loop terminating... [2023-10-13 00:10:14,477][44995] Stopping RolloutWorker_w2... [2023-10-13 00:10:14,476][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... [2023-10-13 00:10:14,477][44995] Loop rollout_proc2_evt_loop terminating... [2023-10-13 00:10:14,477][44999] Stopping RolloutWorker_w6... [2023-10-13 00:10:14,477][44999] Loop rollout_proc6_evt_loop terminating... [2023-10-13 00:10:14,478][44998] Stopping RolloutWorker_w5... [2023-10-13 00:10:14,478][44998] Loop rollout_proc5_evt_loop terminating... [2023-10-13 00:10:14,479][45001] Stopping RolloutWorker_w8... [2023-10-13 00:10:14,479][45000] Stopping RolloutWorker_w7... [2023-10-13 00:10:14,479][45001] Loop rollout_proc8_evt_loop terminating... [2023-10-13 00:10:14,479][45000] Loop rollout_proc7_evt_loop terminating... [2023-10-13 00:10:14,479][44997] Stopping RolloutWorker_w4... [2023-10-13 00:10:14,480][44997] Loop rollout_proc4_evt_loop terminating... [2023-10-13 00:10:14,480][44996] Stopping RolloutWorker_w3... [2023-10-13 00:10:14,480][44996] Loop rollout_proc3_evt_loop terminating... [2023-10-13 00:10:14,480][44991] Stopping RolloutWorker_w1... [2023-10-13 00:10:14,480][45002] Stopping RolloutWorker_w9... [2023-10-13 00:10:14,481][45004] Stopping RolloutWorker_w11... [2023-10-13 00:10:14,481][44991] Loop rollout_proc1_evt_loop terminating... [2023-10-13 00:10:14,481][45002] Loop rollout_proc9_evt_loop terminating... [2023-10-13 00:10:14,481][45004] Loop rollout_proc11_evt_loop terminating... [2023-10-13 00:10:14,482][45003] Stopping RolloutWorker_w10... [2023-10-13 00:10:14,477][44518] Stopping Batcher_0... [2023-10-13 00:10:14,483][45003] Loop rollout_proc10_evt_loop terminating... [2023-10-13 00:10:14,483][44993] Stopping RolloutWorker_w0... [2023-10-13 00:10:14,483][45005] Stopping RolloutWorker_w13... [2023-10-13 00:10:14,483][44993] Loop rollout_proc0_evt_loop terminating... [2023-10-13 00:10:14,484][45005] Loop rollout_proc13_evt_loop terminating... [2023-10-13 00:10:14,483][43579] Component RolloutWorker_w12 stopped! [2023-10-13 00:10:14,484][45784] Stopping RolloutWorker_w15... [2023-10-13 00:10:14,484][45784] Loop rollout_proc15_evt_loop terminating... [2023-10-13 00:10:14,485][43579] Component RolloutWorker_w2 stopped! [2023-10-13 00:10:14,486][43579] Component Batcher_0 stopped! [2023-10-13 00:10:14,486][45783] Stopping RolloutWorker_w14... [2023-10-13 00:10:14,487][45783] Loop rollout_proc14_evt_loop terminating... [2023-10-13 00:10:14,487][43579] Component RolloutWorker_w6 stopped! [2023-10-13 00:10:14,487][43579] Component RolloutWorker_w5 stopped! [2023-10-13 00:10:14,488][43579] Component RolloutWorker_w8 stopped! [2023-10-13 00:10:14,488][43579] Component RolloutWorker_w7 stopped! [2023-10-13 00:10:14,489][43579] Component RolloutWorker_w4 stopped! [2023-10-13 00:10:14,489][43579] Component RolloutWorker_w3 stopped! [2023-10-13 00:10:14,489][43579] Component RolloutWorker_w1 stopped! [2023-10-13 00:10:14,490][43579] Component RolloutWorker_w9 stopped! [2023-10-13 00:10:14,490][43579] Component RolloutWorker_w11 stopped! [2023-10-13 00:10:14,490][43579] Component RolloutWorker_w10 stopped! [2023-10-13 00:10:14,491][43579] Component RolloutWorker_w0 stopped! [2023-10-13 00:10:14,491][43579] Component RolloutWorker_w13 stopped! [2023-10-13 00:10:14,492][43579] Component RolloutWorker_w15 stopped! [2023-10-13 00:10:14,492][43579] Component RolloutWorker_w14 stopped! [2023-10-13 00:10:14,492][43579] Component Batcher_1 stopped! [2023-10-13 00:10:14,488][44583] Stopping Batcher_1... [2023-10-13 00:10:14,502][44959] Weights refcount: 2 0 [2023-10-13 00:10:14,504][44958] Weights refcount: 2 0 [2023-10-13 00:10:14,493][44518] Loop batcher_evt_loop terminating... [2023-10-13 00:10:14,507][44959] Stopping InferenceWorker_p1-w0... [2023-10-13 00:10:14,507][44958] Stopping InferenceWorker_p0-w0... [2023-10-13 00:10:14,507][44959] Loop inference_proc1-0_evt_loop terminating... [2023-10-13 00:10:14,507][43579] Component InferenceWorker_p1-w0 stopped! [2023-10-13 00:10:14,507][44958] Loop inference_proc0-0_evt_loop terminating... [2023-10-13 00:10:14,508][43579] Component InferenceWorker_p0-w0 stopped! [2023-10-13 00:10:14,508][44583] Loop batcher_evt_loop terminating... [2023-10-13 00:10:14,510][44583] Removing ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000097184_99516416.pth [2023-10-13 00:10:14,515][44583] Saving ./train_atari/atari_krull_APPO/checkpoint_p1/checkpoint_000098144_100499456.pth... [2023-10-13 00:10:14,524][44518] Removing ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000096736_99057664.pth [2023-10-13 00:10:14,531][44518] Saving ./train_atari/atari_krull_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... [2023-10-13 00:10:14,555][44583] Stopping LearnerWorker_p1... [2023-10-13 00:10:14,555][44583] Loop learner_proc1_evt_loop terminating... [2023-10-13 00:10:14,555][43579] Component LearnerWorker_p1 stopped! [2023-10-13 00:10:14,583][44518] Stopping LearnerWorker_p0... [2023-10-13 00:10:14,584][43579] Component LearnerWorker_p0 stopped! [2023-10-13 00:10:14,584][44518] Loop learner_proc0_evt_loop terminating... [2023-10-13 00:10:14,584][43579] Waiting for process learner_proc0 to stop... [2023-10-13 00:10:15,523][43579] Waiting for process learner_proc1 to stop... [2023-10-13 00:10:15,524][43579] Waiting for process inference_proc0-0 to join... [2023-10-13 00:10:15,525][43579] Waiting for process inference_proc1-0 to join... [2023-10-13 00:10:15,526][43579] Waiting for process rollout_proc0 to join... [2023-10-13 00:10:15,527][43579] Waiting for process rollout_proc1 to join... [2023-10-13 00:10:15,527][43579] Waiting for process rollout_proc2 to join... [2023-10-13 00:10:15,528][43579] Waiting for process rollout_proc3 to join... [2023-10-13 00:10:15,529][43579] Waiting for process rollout_proc4 to join... [2023-10-13 00:10:15,529][43579] Waiting for process rollout_proc5 to join... [2023-10-13 00:10:15,530][43579] Waiting for process rollout_proc6 to join... [2023-10-13 00:10:15,530][43579] Waiting for process rollout_proc7 to join... [2023-10-13 00:10:15,531][43579] Waiting for process rollout_proc8 to join... [2023-10-13 00:10:15,531][43579] Waiting for process rollout_proc9 to join... [2023-10-13 00:10:15,532][43579] Waiting for process rollout_proc10 to join... [2023-10-13 00:10:15,533][43579] Waiting for process rollout_proc11 to join... [2023-10-13 00:10:15,533][43579] Waiting for process rollout_proc12 to join... [2023-10-13 00:10:15,534][43579] Waiting for process rollout_proc13 to join... [2023-10-13 00:10:15,534][43579] Waiting for process rollout_proc14 to join... [2023-10-13 00:10:15,535][43579] Waiting for process rollout_proc15 to join... [2023-10-13 00:10:15,535][43579] Batcher 0 profile tree view: batching: 167.9519, releasing_batches: 0.0881 [2023-10-13 00:10:15,536][43579] Batcher 1 profile tree view: batching: 168.9503, releasing_batches: 0.0892 [2023-10-13 00:10:15,536][43579] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0002 wait_policy_total: 2874.0363 update_model: 203.8588 weight_update: 0.0010 one_step: 0.0022 handle_policy_step: 11468.9361 deserialize: 64.5398, stack: 190.3927, obs_to_device_normalize: 2555.6878, forward: 5197.2318, prepare_outputs: 2475.8560, send_messages: 473.8813 [2023-10-13 00:10:15,536][43579] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 2921.7096 update_model: 207.9405 weight_update: 0.0008 one_step: 0.0025 handle_policy_step: 11426.1410 deserialize: 65.0077, stack: 195.8426, obs_to_device_normalize: 2560.0377, forward: 5175.1228, prepare_outputs: 2448.2154, send_messages: 475.5131 [2023-10-13 00:10:15,537][43579] Learner 0 profile tree view: misc: 0.0177, prepare_batch: 266.8049 train: 3651.5078 epoch_init: 0.1899, minibatch_init: 13.0591, losses_postprocess: 900.5786, kl_divergence: 32.4654, update: 384.5534, after_optimizer: 2134.7086 calculate_losses: 169.2890 losses_init: 0.3859, forward_head: 59.3771, bptt_initial: 1.4508, bptt: 1.8772, tail: 37.9357, advantages_returns: 11.0667, losses: 43.3790 [2023-10-13 00:10:15,537][43579] Learner 1 profile tree view: misc: 0.0193, prepare_batch: 268.0872 train: 3636.4293 epoch_init: 0.1941, minibatch_init: 13.4627, losses_postprocess: 898.6108, kl_divergence: 31.9779, update: 383.5622, after_optimizer: 2121.3180 calculate_losses: 170.6709 losses_init: 0.3938, forward_head: 59.4810, bptt_initial: 1.4256, bptt: 1.7737, tail: 38.5658, advantages_returns: 11.1653, losses: 44.1412 [2023-10-13 00:10:15,538][43579] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.2159, enqueue_policy_requests: 398.3818, process_policy_outputs: 190.8312, env_step: 7976.9649, finalize_trajectories: 3.4422, complete_rollouts: 2.9045 post_env_step: 371.4672 process_env_step: 83.0545 [2023-10-13 00:10:15,538][43579] RolloutWorker_w15 profile tree view: wait_for_trajectories: 1.2106, enqueue_policy_requests: 400.5402, process_policy_outputs: 190.0021, env_step: 8090.5725, finalize_trajectories: 3.5351, complete_rollouts: 2.8871 post_env_step: 368.8388 process_env_step: 83.3488 [2023-10-13 00:10:15,538][43579] Loop Runner_EvtLoop terminating... [2023-10-13 00:10:15,539][43579] Runner profile tree view: main_loop: 15264.4374 [2023-10-13 00:10:15,539][43579] Collected {0: 100007936, 1: 100499456}, FPS: 13135.6