[2023-09-19 10:11:44,794][36400] Saving configuration to ./train_dir/Humanoid/config.json... [2023-09-19 10:11:44,795][36400] Rollout worker 0 uses device cpu [2023-09-19 10:11:44,795][36400] Rollout worker 1 uses device cpu [2023-09-19 10:11:44,796][36400] Rollout worker 2 uses device cpu [2023-09-19 10:11:44,796][36400] Rollout worker 3 uses device cpu [2023-09-19 10:11:44,796][36400] Rollout worker 4 uses device cpu [2023-09-19 10:11:44,796][36400] Rollout worker 5 uses device cpu [2023-09-19 10:11:44,796][36400] Rollout worker 6 uses device cpu [2023-09-19 10:11:44,796][36400] Rollout worker 7 uses device cpu [2023-09-19 10:11:44,796][36400] In synchronous mode, we only accumulate one batch. Setting num_batches_to_accumulate to 1 [2023-09-19 10:11:44,841][36400] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:11:44,841][36400] InferenceWorker_p0-w0: min num requests: 2 [2023-09-19 10:11:44,866][36400] Starting all processes... [2023-09-19 10:11:44,866][36400] Starting process learner_proc0 [2023-09-19 10:11:44,868][36400] Starting all processes... [2023-09-19 10:11:44,872][36400] Starting process inference_proc0-0 [2023-09-19 10:11:44,873][36400] Starting process rollout_proc0 [2023-09-19 10:11:44,873][36400] Starting process rollout_proc1 [2023-09-19 10:11:44,873][36400] Starting process rollout_proc2 [2023-09-19 10:11:44,873][36400] Starting process rollout_proc3 [2023-09-19 10:11:44,875][36400] Starting process rollout_proc4 [2023-09-19 10:11:44,875][36400] Starting process rollout_proc5 [2023-09-19 10:11:44,877][36400] Starting process rollout_proc6 [2023-09-19 10:11:44,878][36400] Starting process rollout_proc7 [2023-09-19 10:11:46,682][36856] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:11:46,682][36856] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-09-19 10:11:46,686][36862] Worker 3 uses CPU cores [12, 13, 14, 15] [2023-09-19 10:11:46,700][36856] Num visible devices: 1 [2023-09-19 10:11:46,710][36860] Worker 4 uses CPU cores [16, 17, 18, 19] [2023-09-19 10:11:46,718][36861] Worker 5 uses CPU cores [20, 21, 22, 23] [2023-09-19 10:11:46,722][36842] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:11:46,723][36842] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-09-19 10:11:46,731][36864] Worker 7 uses CPU cores [28, 29, 30, 31] [2023-09-19 10:11:46,735][36857] Worker 0 uses CPU cores [0, 1, 2, 3] [2023-09-19 10:11:46,740][36858] Worker 1 uses CPU cores [4, 5, 6, 7] [2023-09-19 10:11:46,742][36842] Num visible devices: 1 [2023-09-19 10:11:46,772][36842] Starting seed is not provided [2023-09-19 10:11:46,772][36842] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:11:46,772][36842] Initializing actor-critic model on device cuda:0 [2023-09-19 10:11:46,773][36842] RunningMeanStd input shape: (376,) [2023-09-19 10:11:46,773][36842] RunningMeanStd input shape: (1,) [2023-09-19 10:11:46,815][36859] Worker 2 uses CPU cores [8, 9, 10, 11] [2023-09-19 10:11:46,833][36863] Worker 6 uses CPU cores [24, 25, 26, 27] [2023-09-19 10:11:46,849][36842] Created Actor Critic model with architecture: [2023-09-19 10:11:46,849][36842] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): MlpEncoder( (mlp_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=Tanh) (2): RecursiveScriptModule(original_name=Linear) (3): RecursiveScriptModule(original_name=Tanh) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=64, out_features=1, bias=True) (action_parameterization): ActionParameterizationContinuousNonAdaptiveStddev( (distribution_linear): Linear(in_features=64, out_features=17, bias=True) ) ) [2023-09-19 10:11:47,405][36842] Using optimizer [2023-09-19 10:11:47,406][36842] No checkpoints found [2023-09-19 10:11:47,406][36842] Did not load from checkpoint, starting from scratch! [2023-09-19 10:11:47,406][36842] Initialized policy 0 weights for model version 0 [2023-09-19 10:11:47,408][36842] LearnerWorker_p0 finished initialization! [2023-09-19 10:11:47,408][36842] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:11:47,978][36856] RunningMeanStd input shape: (376,) [2023-09-19 10:11:47,979][36856] RunningMeanStd input shape: (1,) [2023-09-19 10:11:48,011][36400] Inference worker 0-0 is ready! [2023-09-19 10:11:48,012][36400] All inference workers are ready! Signal rollout workers to start! [2023-09-19 10:11:48,113][36862] Decorrelating experience for 0 frames... [2023-09-19 10:11:48,113][36862] Decorrelating experience for 64 frames... [2023-09-19 10:11:48,117][36864] Decorrelating experience for 0 frames... [2023-09-19 10:11:48,118][36864] Decorrelating experience for 64 frames... [2023-09-19 10:11:48,137][36863] Decorrelating experience for 0 frames... [2023-09-19 10:11:48,138][36863] Decorrelating experience for 64 frames... [2023-09-19 10:11:48,138][36859] Decorrelating experience for 0 frames... [2023-09-19 10:11:48,139][36859] Decorrelating experience for 64 frames... [2023-09-19 10:11:48,157][36861] Decorrelating experience for 0 frames... [2023-09-19 10:11:48,157][36861] Decorrelating experience for 64 frames... [2023-09-19 10:11:48,159][36864] Decorrelating experience for 128 frames... [2023-09-19 10:11:48,160][36860] Decorrelating experience for 0 frames... [2023-09-19 10:11:48,160][36857] Decorrelating experience for 0 frames... [2023-09-19 10:11:48,160][36862] Decorrelating experience for 128 frames... [2023-09-19 10:11:48,161][36857] Decorrelating experience for 64 frames... [2023-09-19 10:11:48,161][36860] Decorrelating experience for 64 frames... [2023-09-19 10:11:48,163][36858] Decorrelating experience for 0 frames... [2023-09-19 10:11:48,164][36858] Decorrelating experience for 64 frames... [2023-09-19 10:11:48,199][36861] Decorrelating experience for 128 frames... [2023-09-19 10:11:48,205][36858] Decorrelating experience for 128 frames... [2023-09-19 10:11:48,206][36863] Decorrelating experience for 128 frames... [2023-09-19 10:11:48,206][36859] Decorrelating experience for 128 frames... [2023-09-19 10:11:48,232][36860] Decorrelating experience for 128 frames... [2023-09-19 10:11:48,236][36857] Decorrelating experience for 128 frames... [2023-09-19 10:11:48,240][36864] Decorrelating experience for 192 frames... [2023-09-19 10:11:48,244][36862] Decorrelating experience for 192 frames... [2023-09-19 10:11:48,280][36861] Decorrelating experience for 192 frames... [2023-09-19 10:11:48,293][36858] Decorrelating experience for 192 frames... [2023-09-19 10:11:48,343][36863] Decorrelating experience for 192 frames... [2023-09-19 10:11:48,346][36859] Decorrelating experience for 192 frames... [2023-09-19 10:11:48,367][36857] Decorrelating experience for 192 frames... [2023-09-19 10:11:48,372][36860] Decorrelating experience for 192 frames... [2023-09-19 10:11:48,380][36864] Decorrelating experience for 256 frames... [2023-09-19 10:11:48,382][36862] Decorrelating experience for 256 frames... [2023-09-19 10:11:48,428][36858] Decorrelating experience for 256 frames... [2023-09-19 10:11:48,431][36861] Decorrelating experience for 256 frames... [2023-09-19 10:11:48,545][36864] Decorrelating experience for 320 frames... [2023-09-19 10:11:48,545][36862] Decorrelating experience for 320 frames... [2023-09-19 10:11:48,550][36863] Decorrelating experience for 256 frames... [2023-09-19 10:11:48,556][36857] Decorrelating experience for 256 frames... [2023-09-19 10:11:48,560][36859] Decorrelating experience for 256 frames... [2023-09-19 10:11:48,563][36860] Decorrelating experience for 256 frames... [2023-09-19 10:11:48,581][36858] Decorrelating experience for 320 frames... [2023-09-19 10:11:48,596][36861] Decorrelating experience for 320 frames... [2023-09-19 10:11:48,723][36860] Decorrelating experience for 320 frames... [2023-09-19 10:11:48,724][36857] Decorrelating experience for 320 frames... [2023-09-19 10:11:48,725][36859] Decorrelating experience for 320 frames... [2023-09-19 10:11:48,725][36863] Decorrelating experience for 320 frames... [2023-09-19 10:11:48,746][36864] Decorrelating experience for 384 frames... [2023-09-19 10:11:48,764][36862] Decorrelating experience for 384 frames... [2023-09-19 10:11:48,792][36858] Decorrelating experience for 384 frames... [2023-09-19 10:11:48,799][36861] Decorrelating experience for 384 frames... [2023-09-19 10:11:48,919][36863] Decorrelating experience for 384 frames... [2023-09-19 10:11:48,921][36860] Decorrelating experience for 384 frames... [2023-09-19 10:11:48,925][36857] Decorrelating experience for 384 frames... [2023-09-19 10:11:48,942][36859] Decorrelating experience for 384 frames... [2023-09-19 10:11:48,993][36862] Decorrelating experience for 448 frames... [2023-09-19 10:11:48,995][36864] Decorrelating experience for 448 frames... [2023-09-19 10:11:49,026][36858] Decorrelating experience for 448 frames... [2023-09-19 10:11:49,050][36861] Decorrelating experience for 448 frames... [2023-09-19 10:11:49,152][36857] Decorrelating experience for 448 frames... [2023-09-19 10:11:49,153][36860] Decorrelating experience for 448 frames... [2023-09-19 10:11:49,155][36863] Decorrelating experience for 448 frames... [2023-09-19 10:11:49,180][36859] Decorrelating experience for 448 frames... [2023-09-19 10:11:51,169][36400] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 4096. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-09-19 10:11:51,169][36400] Avg episode reward: [(0, '66.846')] [2023-09-19 10:11:56,169][36400] Fps is (10 sec: 4915.2, 60 sec: 4915.2, 300 sec: 4915.2). Total num frames: 24576. Throughput: 0: 1765.6. Samples: 12924. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:11:56,169][36400] Avg episode reward: [(0, '113.634')] [2023-09-19 10:11:56,175][36842] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000048_24576.pth... [2023-09-19 10:11:58,183][36856] Updated weights for policy 0, policy_version 80 (0.0013) [2023-09-19 10:12:01,169][36400] Fps is (10 sec: 5734.4, 60 sec: 5734.4, 300 sec: 5734.4). Total num frames: 57344. Throughput: 0: 4992.4. Samples: 54020. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:12:01,169][36400] Avg episode reward: [(0, '235.291')] [2023-09-19 10:12:04,060][36856] Updated weights for policy 0, policy_version 160 (0.0013) [2023-09-19 10:12:04,835][36400] Heartbeat connected on Batcher_0 [2023-09-19 10:12:04,838][36400] Heartbeat connected on LearnerWorker_p0 [2023-09-19 10:12:04,844][36400] Heartbeat connected on InferenceWorker_p0-w0 [2023-09-19 10:12:04,849][36400] Heartbeat connected on RolloutWorker_w0 [2023-09-19 10:12:04,853][36400] Heartbeat connected on RolloutWorker_w1 [2023-09-19 10:12:04,855][36400] Heartbeat connected on RolloutWorker_w2 [2023-09-19 10:12:04,858][36400] Heartbeat connected on RolloutWorker_w3 [2023-09-19 10:12:04,862][36400] Heartbeat connected on RolloutWorker_w4 [2023-09-19 10:12:04,864][36400] Heartbeat connected on RolloutWorker_w5 [2023-09-19 10:12:04,868][36400] Heartbeat connected on RolloutWorker_w6 [2023-09-19 10:12:04,872][36400] Heartbeat connected on RolloutWorker_w7 [2023-09-19 10:12:06,168][36400] Fps is (10 sec: 6963.3, 60 sec: 6280.6, 300 sec: 6280.6). Total num frames: 94208. Throughput: 0: 6031.2. Samples: 94564. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:12:06,169][36400] Avg episode reward: [(0, '271.600')] [2023-09-19 10:12:10,187][36856] Updated weights for policy 0, policy_version 240 (0.0014) [2023-09-19 10:12:11,169][36400] Fps is (10 sec: 6963.1, 60 sec: 6348.7, 300 sec: 6348.7). Total num frames: 126976. Throughput: 0: 5585.7. Samples: 115812. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:12:11,169][36400] Avg episode reward: [(0, '295.136')] [2023-09-19 10:12:11,175][36842] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000248_126976.pth... [2023-09-19 10:12:11,181][36842] Saving new best policy, reward=295.136! [2023-09-19 10:12:16,168][36400] Fps is (10 sec: 6144.0, 60 sec: 6226.0, 300 sec: 6226.0). Total num frames: 155648. Throughput: 0: 5898.3. Samples: 151552. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:12:16,169][36400] Avg episode reward: [(0, '321.769')] [2023-09-19 10:12:16,169][36842] Saving new best policy, reward=321.769! [2023-09-19 10:12:16,954][36856] Updated weights for policy 0, policy_version 320 (0.0015) [2023-09-19 10:12:21,168][36400] Fps is (10 sec: 6553.8, 60 sec: 6417.1, 300 sec: 6417.1). Total num frames: 192512. Throughput: 0: 6306.7. Samples: 193296. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:12:21,169][36400] Avg episode reward: [(0, '347.002')] [2023-09-19 10:12:21,170][36842] Saving new best policy, reward=347.002! [2023-09-19 10:12:22,788][36856] Updated weights for policy 0, policy_version 400 (0.0014) [2023-09-19 10:12:26,168][36400] Fps is (10 sec: 6963.2, 60 sec: 6436.6, 300 sec: 6436.6). Total num frames: 225280. Throughput: 0: 5970.3. Samples: 213056. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:12:26,169][36400] Avg episode reward: [(0, '360.618')] [2023-09-19 10:12:26,174][36842] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000440_225280.pth... [2023-09-19 10:12:26,181][36842] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000048_24576.pth [2023-09-19 10:12:26,182][36842] Saving new best policy, reward=360.618! [2023-09-19 10:12:27,186][36400] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 36400], exiting... [2023-09-19 10:12:27,187][36400] Runner profile tree view: main_loop: 42.3215 [2023-09-19 10:12:27,188][36400] Collected {0: 233472}, FPS: 5516.6 [2023-09-19 10:12:27,188][36842] Stopping Batcher_0... [2023-09-19 10:12:27,189][36842] Loop batcher_evt_loop terminating... [2023-09-19 10:12:27,189][36842] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000456_233472.pth... [2023-09-19 10:12:27,190][36862] Stopping RolloutWorker_w3... [2023-09-19 10:12:27,190][36864] Stopping RolloutWorker_w7... [2023-09-19 10:12:27,190][36861] Stopping RolloutWorker_w5... [2023-09-19 10:12:27,190][36864] Loop rollout_proc7_evt_loop terminating... [2023-09-19 10:12:27,190][36862] Loop rollout_proc3_evt_loop terminating... [2023-09-19 10:12:27,190][36861] Loop rollout_proc5_evt_loop terminating... [2023-09-19 10:12:27,191][36857] Stopping RolloutWorker_w0... [2023-09-19 10:12:27,191][36858] Stopping RolloutWorker_w1... [2023-09-19 10:12:27,191][36857] Loop rollout_proc0_evt_loop terminating... [2023-09-19 10:12:27,191][36858] Loop rollout_proc1_evt_loop terminating... [2023-09-19 10:12:27,192][36860] Stopping RolloutWorker_w4... [2023-09-19 10:12:27,192][36860] Loop rollout_proc4_evt_loop terminating... [2023-09-19 10:12:27,192][36859] Stopping RolloutWorker_w2... [2023-09-19 10:12:27,192][36859] Loop rollout_proc2_evt_loop terminating... [2023-09-19 10:12:27,193][36863] Stopping RolloutWorker_w6... [2023-09-19 10:12:27,194][36863] Loop rollout_proc6_evt_loop terminating... [2023-09-19 10:12:27,196][36842] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000248_126976.pth [2023-09-19 10:12:27,197][36842] Stopping LearnerWorker_p0... [2023-09-19 10:12:27,197][36842] Loop learner_proc0_evt_loop terminating... [2023-09-19 10:12:27,203][36856] Weights refcount: 2 0 [2023-09-19 10:12:27,204][36856] Stopping InferenceWorker_p0-w0... [2023-09-19 10:12:27,207][36856] Loop inference_proc0-0_evt_loop terminating... [2023-09-19 10:13:45,347][45164] Saving configuration to ./train_dir/Humanoid/config.json... [2023-09-19 10:13:45,349][45164] Rollout worker 0 uses device cpu [2023-09-19 10:13:45,349][45164] Rollout worker 1 uses device cpu [2023-09-19 10:13:45,350][45164] Rollout worker 2 uses device cpu [2023-09-19 10:13:45,350][45164] Rollout worker 3 uses device cpu [2023-09-19 10:13:45,351][45164] Rollout worker 4 uses device cpu [2023-09-19 10:13:45,351][45164] Rollout worker 5 uses device cpu [2023-09-19 10:13:45,352][45164] Rollout worker 6 uses device cpu [2023-09-19 10:13:45,353][45164] Rollout worker 7 uses device cpu [2023-09-19 10:13:45,354][45164] In synchronous mode, we only accumulate one batch. Setting num_batches_to_accumulate to 1 [2023-09-19 10:13:45,410][45164] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:13:45,410][45164] InferenceWorker_p0-w0: min num requests: 1 [2023-09-19 10:13:45,413][45164] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-09-19 10:13:45,414][45164] InferenceWorker_p1-w0: min num requests: 1 [2023-09-19 10:13:45,438][45164] Starting all processes... [2023-09-19 10:13:45,439][45164] Starting process learner_proc0 [2023-09-19 10:13:45,442][45164] Starting process learner_proc1 [2023-09-19 10:13:45,488][45164] Starting all processes... [2023-09-19 10:13:45,495][45164] Starting process inference_proc0-0 [2023-09-19 10:13:45,496][45164] Starting process inference_proc1-0 [2023-09-19 10:13:45,496][45164] Starting process rollout_proc0 [2023-09-19 10:13:45,496][45164] Starting process rollout_proc1 [2023-09-19 10:13:45,497][45164] Starting process rollout_proc2 [2023-09-19 10:13:45,497][45164] Starting process rollout_proc3 [2023-09-19 10:13:45,499][45164] Starting process rollout_proc4 [2023-09-19 10:13:45,500][45164] Starting process rollout_proc5 [2023-09-19 10:13:45,500][45164] Starting process rollout_proc6 [2023-09-19 10:13:45,517][45164] Starting process rollout_proc7 [2023-09-19 10:13:47,413][45898] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-09-19 10:13:47,413][45898] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-09-19 10:13:47,426][45899] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:13:47,427][45899] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-09-19 10:13:47,431][45898] Num visible devices: 1 [2023-09-19 10:13:47,433][45902] Worker 2 uses CPU cores [8, 9, 10, 11] [2023-09-19 10:13:47,463][45899] Num visible devices: 1 [2023-09-19 10:13:47,471][45903] Worker 4 uses CPU cores [16, 17, 18, 19] [2023-09-19 10:13:47,475][45906] Worker 6 uses CPU cores [24, 25, 26, 27] [2023-09-19 10:13:47,481][45901] Worker 1 uses CPU cores [4, 5, 6, 7] [2023-09-19 10:13:47,539][45904] Worker 3 uses CPU cores [12, 13, 14, 15] [2023-09-19 10:13:47,654][45907] Worker 7 uses CPU cores [28, 29, 30, 31] [2023-09-19 10:13:47,750][45905] Worker 5 uses CPU cores [20, 21, 22, 23] [2023-09-19 10:13:47,788][45811] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-09-19 10:13:47,788][45811] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-09-19 10:13:47,806][45811] Num visible devices: 1 [2023-09-19 10:13:47,807][45810] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:13:47,807][45810] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-09-19 10:13:47,816][45900] Worker 0 uses CPU cores [0, 1, 2, 3] [2023-09-19 10:13:47,826][45810] Num visible devices: 1 [2023-09-19 10:13:47,848][45811] Starting seed is not provided [2023-09-19 10:13:47,848][45811] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-09-19 10:13:47,848][45811] Initializing actor-critic model on device cuda:0 [2023-09-19 10:13:47,848][45810] Starting seed is not provided [2023-09-19 10:13:47,848][45810] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:13:47,848][45810] Initializing actor-critic model on device cuda:0 [2023-09-19 10:13:47,849][45811] RunningMeanStd input shape: (376,) [2023-09-19 10:13:47,849][45810] RunningMeanStd input shape: (376,) [2023-09-19 10:13:47,849][45811] RunningMeanStd input shape: (1,) [2023-09-19 10:13:47,849][45810] RunningMeanStd input shape: (1,) [2023-09-19 10:13:47,893][45810] Created Actor Critic model with architecture: [2023-09-19 10:13:47,894][45810] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): MlpEncoder( (mlp_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=Tanh) (2): RecursiveScriptModule(original_name=Linear) (3): RecursiveScriptModule(original_name=Tanh) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=64, out_features=1, bias=True) (action_parameterization): ActionParameterizationContinuousNonAdaptiveStddev( (distribution_linear): Linear(in_features=64, out_features=17, bias=True) ) ) [2023-09-19 10:13:47,901][45811] Created Actor Critic model with architecture: [2023-09-19 10:13:47,901][45811] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): MlpEncoder( (mlp_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=Tanh) (2): RecursiveScriptModule(original_name=Linear) (3): RecursiveScriptModule(original_name=Tanh) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=64, out_features=1, bias=True) (action_parameterization): ActionParameterizationContinuousNonAdaptiveStddev( (distribution_linear): Linear(in_features=64, out_features=17, bias=True) ) ) [2023-09-19 10:13:48,482][45810] Using optimizer [2023-09-19 10:13:48,482][45810] Loading state from checkpoint ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000456_233472.pth... [2023-09-19 10:13:48,488][45810] Loading model from checkpoint [2023-09-19 10:13:48,490][45810] Loaded experiment state at self.train_step=456, self.env_steps=233472 [2023-09-19 10:13:48,490][45811] Using optimizer [2023-09-19 10:13:48,490][45810] Initialized policy 0 weights for model version 456 [2023-09-19 10:13:48,491][45811] No checkpoints found [2023-09-19 10:13:48,491][45811] Did not load from checkpoint, starting from scratch! [2023-09-19 10:13:48,491][45811] Initialized policy 1 weights for model version 0 [2023-09-19 10:13:48,492][45810] LearnerWorker_p0 finished initialization! [2023-09-19 10:13:48,492][45810] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-09-19 10:13:48,492][45811] LearnerWorker_p1 finished initialization! [2023-09-19 10:13:48,493][45811] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-09-19 10:13:49,087][45899] RunningMeanStd input shape: (376,) [2023-09-19 10:13:49,087][45899] RunningMeanStd input shape: (1,) [2023-09-19 10:13:49,099][45898] RunningMeanStd input shape: (376,) [2023-09-19 10:13:49,100][45898] RunningMeanStd input shape: (1,) [2023-09-19 10:13:49,121][45164] Inference worker 0-0 is ready! [2023-09-19 10:13:49,134][45164] Inference worker 1-0 is ready! [2023-09-19 10:13:49,135][45164] All inference workers are ready! Signal rollout workers to start! [2023-09-19 10:13:49,231][45905] Decorrelating experience for 0 frames... [2023-09-19 10:13:49,232][45905] Decorrelating experience for 64 frames... [2023-09-19 10:13:49,232][45901] Decorrelating experience for 0 frames... [2023-09-19 10:13:49,233][45901] Decorrelating experience for 64 frames... [2023-09-19 10:13:49,243][45907] Decorrelating experience for 0 frames... [2023-09-19 10:13:49,244][45904] Decorrelating experience for 0 frames... [2023-09-19 10:13:49,244][45907] Decorrelating experience for 64 frames... [2023-09-19 10:13:49,244][45904] Decorrelating experience for 64 frames... [2023-09-19 10:13:49,254][45902] Decorrelating experience for 0 frames... [2023-09-19 10:13:49,255][45902] Decorrelating experience for 64 frames... [2023-09-19 10:13:49,255][45906] Decorrelating experience for 0 frames... [2023-09-19 10:13:49,256][45906] Decorrelating experience for 64 frames... [2023-09-19 10:13:49,274][45901] Decorrelating experience for 128 frames... [2023-09-19 10:13:49,275][45905] Decorrelating experience for 128 frames... [2023-09-19 10:13:49,285][45907] Decorrelating experience for 128 frames... [2023-09-19 10:13:49,285][45904] Decorrelating experience for 128 frames... [2023-09-19 10:13:49,290][45903] Decorrelating experience for 0 frames... [2023-09-19 10:13:49,290][45900] Decorrelating experience for 0 frames... [2023-09-19 10:13:49,291][45900] Decorrelating experience for 64 frames... [2023-09-19 10:13:49,291][45903] Decorrelating experience for 64 frames... [2023-09-19 10:13:49,309][45906] Decorrelating experience for 128 frames... [2023-09-19 10:13:49,317][45902] Decorrelating experience for 128 frames... [2023-09-19 10:13:49,358][45903] Decorrelating experience for 128 frames... [2023-09-19 10:13:49,359][45901] Decorrelating experience for 192 frames... [2023-09-19 10:13:49,359][45905] Decorrelating experience for 192 frames... [2023-09-19 10:13:49,364][45900] Decorrelating experience for 128 frames... [2023-09-19 10:13:49,367][45907] Decorrelating experience for 192 frames... [2023-09-19 10:13:49,369][45904] Decorrelating experience for 192 frames... [2023-09-19 10:13:49,384][45906] Decorrelating experience for 192 frames... [2023-09-19 10:13:49,402][45902] Decorrelating experience for 192 frames... [2023-09-19 10:13:49,491][45903] Decorrelating experience for 192 frames... [2023-09-19 10:13:49,499][45901] Decorrelating experience for 256 frames... [2023-09-19 10:13:49,501][45905] Decorrelating experience for 256 frames... [2023-09-19 10:13:49,507][45900] Decorrelating experience for 192 frames... [2023-09-19 10:13:49,508][45907] Decorrelating experience for 256 frames... [2023-09-19 10:13:49,521][45906] Decorrelating experience for 256 frames... [2023-09-19 10:13:49,521][45904] Decorrelating experience for 256 frames... [2023-09-19 10:13:49,545][45902] Decorrelating experience for 256 frames... [2023-09-19 10:13:49,661][45905] Decorrelating experience for 320 frames... [2023-09-19 10:13:49,674][45907] Decorrelating experience for 320 frames... [2023-09-19 10:13:49,678][45906] Decorrelating experience for 320 frames... [2023-09-19 10:13:49,696][45901] Decorrelating experience for 320 frames... [2023-09-19 10:13:49,704][45904] Decorrelating experience for 320 frames... [2023-09-19 10:13:49,708][45902] Decorrelating experience for 320 frames... [2023-09-19 10:13:49,729][45903] Decorrelating experience for 256 frames... [2023-09-19 10:13:49,737][45900] Decorrelating experience for 256 frames... [2023-09-19 10:13:49,868][45905] Decorrelating experience for 384 frames... [2023-09-19 10:13:49,873][45906] Decorrelating experience for 384 frames... [2023-09-19 10:13:49,879][45907] Decorrelating experience for 384 frames... [2023-09-19 10:13:49,893][45904] Decorrelating experience for 384 frames... [2023-09-19 10:13:49,895][45901] Decorrelating experience for 384 frames... [2023-09-19 10:13:49,920][45902] Decorrelating experience for 384 frames... [2023-09-19 10:13:49,999][45903] Decorrelating experience for 320 frames... [2023-09-19 10:13:50,016][45900] Decorrelating experience for 320 frames... [2023-09-19 10:13:50,109][45906] Decorrelating experience for 448 frames... [2023-09-19 10:13:50,116][45905] Decorrelating experience for 448 frames... [2023-09-19 10:13:50,122][45907] Decorrelating experience for 448 frames... [2023-09-19 10:13:50,131][45904] Decorrelating experience for 448 frames... [2023-09-19 10:13:50,148][45901] Decorrelating experience for 448 frames... [2023-09-19 10:13:50,165][45902] Decorrelating experience for 448 frames... [2023-09-19 10:13:50,210][45903] Decorrelating experience for 384 frames... [2023-09-19 10:13:50,221][45900] Decorrelating experience for 384 frames... [2023-09-19 10:13:50,443][45903] Decorrelating experience for 448 frames... [2023-09-19 10:13:50,490][45900] Decorrelating experience for 448 frames... [2023-09-19 10:13:51,681][45164] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 233472. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-09-19 10:13:51,682][45164] Avg episode reward: [(0, '171.567'), (1, '66.937')] [2023-09-19 10:13:56,681][45164] Fps is (10 sec: 4915.3, 60 sec: 4915.3, 300 sec: 4915.3). Total num frames: 258048. Throughput: 0: 1761.6, 1: 1769.2. Samples: 17654. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:13:56,682][45164] Avg episode reward: [(0, '372.657'), (1, '84.499')] [2023-09-19 10:13:56,687][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000024_12288.pth... [2023-09-19 10:13:56,688][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000480_245760.pth... [2023-09-19 10:13:56,695][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000440_225280.pth [2023-09-19 10:13:56,695][45810] Saving new best policy, reward=372.657! [2023-09-19 10:14:01,681][45164] Fps is (10 sec: 5734.4, 60 sec: 5734.4, 300 sec: 5734.4). Total num frames: 290816. Throughput: 0: 2943.4, 1: 2949.8. Samples: 58932. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:14:01,682][45164] Avg episode reward: [(0, '374.414'), (1, '142.065')] [2023-09-19 10:14:01,684][45810] Saving new best policy, reward=374.414! [2023-09-19 10:14:04,472][45899] Updated weights for policy 0, policy_version 536 (0.0015) [2023-09-19 10:14:04,472][45898] Updated weights for policy 1, policy_version 80 (0.0014) [2023-09-19 10:14:05,396][45164] Heartbeat connected on Batcher_0 [2023-09-19 10:14:05,399][45164] Heartbeat connected on LearnerWorker_p0 [2023-09-19 10:14:05,403][45164] Heartbeat connected on Batcher_1 [2023-09-19 10:14:05,406][45164] Heartbeat connected on LearnerWorker_p1 [2023-09-19 10:14:05,417][45164] Heartbeat connected on RolloutWorker_w0 [2023-09-19 10:14:05,420][45164] Heartbeat connected on RolloutWorker_w1 [2023-09-19 10:14:05,423][45164] Heartbeat connected on RolloutWorker_w2 [2023-09-19 10:14:05,426][45164] Heartbeat connected on RolloutWorker_w3 [2023-09-19 10:14:05,429][45164] Heartbeat connected on RolloutWorker_w4 [2023-09-19 10:14:05,432][45164] Heartbeat connected on RolloutWorker_w5 [2023-09-19 10:14:05,435][45164] Heartbeat connected on RolloutWorker_w6 [2023-09-19 10:14:05,437][45164] Heartbeat connected on RolloutWorker_w7 [2023-09-19 10:14:05,454][45164] Heartbeat connected on InferenceWorker_p0-w0 [2023-09-19 10:14:05,454][45164] Heartbeat connected on InferenceWorker_p1-w0 [2023-09-19 10:14:06,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6007.5, 300 sec: 6007.5). Total num frames: 323584. Throughput: 0: 2693.9, 1: 2698.0. Samples: 80878. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:14:06,682][45164] Avg episode reward: [(0, '388.542'), (1, '191.898')] [2023-09-19 10:14:06,683][45810] Saving new best policy, reward=388.542! [2023-09-19 10:14:11,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.5, 300 sec: 6553.5). Total num frames: 364544. Throughput: 0: 3037.3, 1: 3039.9. Samples: 121544. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:14:11,683][45164] Avg episode reward: [(0, '389.184'), (1, '237.269')] [2023-09-19 10:14:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000584_299008.pth... [2023-09-19 10:14:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000128_65536.pth... [2023-09-19 10:14:11,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000456_233472.pth [2023-09-19 10:14:11,698][45810] Saving new best policy, reward=389.184! [2023-09-19 10:14:15,972][45898] Updated weights for policy 1, policy_version 160 (0.0011) [2023-09-19 10:14:15,974][45899] Updated weights for policy 0, policy_version 616 (0.0013) [2023-09-19 10:14:16,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 397312. Throughput: 0: 3299.2, 1: 3301.3. Samples: 165012. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:14:16,682][45164] Avg episode reward: [(0, '437.797'), (1, '257.182')] [2023-09-19 10:14:16,684][45810] Saving new best policy, reward=437.797! [2023-09-19 10:14:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 430080. Throughput: 0: 3087.1, 1: 3089.0. Samples: 185282. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:14:21,682][45164] Avg episode reward: [(0, '430.611'), (1, '263.884')] [2023-09-19 10:14:26,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6787.6, 300 sec: 6787.6). Total num frames: 471040. Throughput: 0: 3285.1, 1: 3286.8. Samples: 230018. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:14:26,683][45164] Avg episode reward: [(0, '438.253'), (1, '278.993')] [2023-09-19 10:14:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000688_352256.pth... [2023-09-19 10:14:26,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000232_118784.pth... [2023-09-19 10:14:26,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000480_245760.pth [2023-09-19 10:14:26,699][45810] Saving new best policy, reward=438.253! [2023-09-19 10:14:26,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000024_12288.pth [2023-09-19 10:14:26,704][45811] Saving new best policy, reward=278.993! [2023-09-19 10:14:27,144][45899] Updated weights for policy 0, policy_version 696 (0.0015) [2023-09-19 10:14:27,145][45898] Updated weights for policy 1, policy_version 240 (0.0013) [2023-09-19 10:14:31,681][45164] Fps is (10 sec: 8192.0, 60 sec: 6963.2, 300 sec: 6963.2). Total num frames: 512000. Throughput: 0: 3469.0, 1: 3470.4. Samples: 277574. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:14:31,682][45164] Avg episode reward: [(0, '423.455'), (1, '298.589')] [2023-09-19 10:14:31,683][45811] Saving new best policy, reward=298.589! [2023-09-19 10:14:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6917.7, 300 sec: 6917.7). Total num frames: 544768. Throughput: 0: 3345.6, 1: 3346.9. Samples: 301166. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:14:36,682][45164] Avg episode reward: [(0, '468.621'), (1, '314.855')] [2023-09-19 10:14:36,684][45810] Saving new best policy, reward=468.621! [2023-09-19 10:14:36,684][45811] Saving new best policy, reward=314.855! [2023-09-19 10:14:37,936][45899] Updated weights for policy 0, policy_version 776 (0.0010) [2023-09-19 10:14:37,937][45898] Updated weights for policy 1, policy_version 320 (0.0016) [2023-09-19 10:14:41,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7045.1, 300 sec: 7045.1). Total num frames: 585728. Throughput: 0: 3640.5, 1: 3640.9. Samples: 345316. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:14:41,682][45164] Avg episode reward: [(0, '462.473'), (1, '297.786')] [2023-09-19 10:14:41,688][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000800_409600.pth... [2023-09-19 10:14:41,688][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000344_176128.pth... [2023-09-19 10:14:41,695][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000584_299008.pth [2023-09-19 10:14:41,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000128_65536.pth [2023-09-19 10:14:46,681][45164] Fps is (10 sec: 8192.1, 60 sec: 7149.4, 300 sec: 7149.4). Total num frames: 626688. Throughput: 0: 3713.8, 1: 3712.8. Samples: 393126. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:14:46,682][45164] Avg episode reward: [(0, '490.587'), (1, '325.263')] [2023-09-19 10:14:46,683][45810] Saving new best policy, reward=490.587! [2023-09-19 10:14:46,683][45811] Saving new best policy, reward=325.263! [2023-09-19 10:14:48,329][45898] Updated weights for policy 1, policy_version 400 (0.0014) [2023-09-19 10:14:48,329][45899] Updated weights for policy 0, policy_version 856 (0.0016) [2023-09-19 10:14:51,681][45164] Fps is (10 sec: 8191.9, 60 sec: 7236.3, 300 sec: 7236.3). Total num frames: 667648. Throughput: 0: 3732.2, 1: 3732.3. Samples: 416782. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:14:51,683][45164] Avg episode reward: [(0, '509.221'), (1, '342.762')] [2023-09-19 10:14:51,684][45810] Saving new best policy, reward=509.221! [2023-09-19 10:14:51,684][45811] Saving new best policy, reward=342.762! [2023-09-19 10:14:56,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7372.8, 300 sec: 7183.8). Total num frames: 700416. Throughput: 0: 3782.9, 1: 3783.1. Samples: 462012. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:14:56,682][45164] Avg episode reward: [(0, '494.347'), (1, '331.691')] [2023-09-19 10:14:56,688][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000912_466944.pth... [2023-09-19 10:14:56,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000456_233472.pth... [2023-09-19 10:14:56,694][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000688_352256.pth [2023-09-19 10:14:56,695][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000232_118784.pth [2023-09-19 10:15:00,045][45898] Updated weights for policy 1, policy_version 480 (0.0011) [2023-09-19 10:15:00,046][45899] Updated weights for policy 0, policy_version 936 (0.0015) [2023-09-19 10:15:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7372.8, 300 sec: 7138.7). Total num frames: 733184. Throughput: 0: 3712.0, 1: 3711.4. Samples: 499064. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:15:01,682][45164] Avg episode reward: [(0, '528.916'), (1, '331.022')] [2023-09-19 10:15:01,683][45810] Saving new best policy, reward=528.916! [2023-09-19 10:15:06,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7372.8, 300 sec: 7099.7). Total num frames: 765952. Throughput: 0: 3676.3, 1: 3675.1. Samples: 516092. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:15:06,682][45164] Avg episode reward: [(0, '507.751'), (1, '350.798')] [2023-09-19 10:15:06,683][45811] Saving new best policy, reward=350.798! [2023-09-19 10:15:11,681][45164] Fps is (10 sec: 6553.5, 60 sec: 7236.3, 300 sec: 7065.6). Total num frames: 798720. Throughput: 0: 3646.8, 1: 3646.8. Samples: 558228. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:15:11,683][45164] Avg episode reward: [(0, '537.704'), (1, '343.602')] [2023-09-19 10:15:11,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000552_282624.pth... [2023-09-19 10:15:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001008_516096.pth... [2023-09-19 10:15:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000344_176128.pth [2023-09-19 10:15:11,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000800_409600.pth [2023-09-19 10:15:11,702][45810] Saving new best policy, reward=537.704! [2023-09-19 10:15:12,645][45899] Updated weights for policy 0, policy_version 1016 (0.0013) [2023-09-19 10:15:12,646][45898] Updated weights for policy 1, policy_version 560 (0.0013) [2023-09-19 10:15:16,681][45164] Fps is (10 sec: 6553.5, 60 sec: 7236.3, 300 sec: 7035.5). Total num frames: 831488. Throughput: 0: 3540.3, 1: 3540.5. Samples: 596210. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:15:16,682][45164] Avg episode reward: [(0, '547.542'), (1, '377.102')] [2023-09-19 10:15:16,683][45811] Saving new best policy, reward=377.102! [2023-09-19 10:15:16,683][45810] Saving new best policy, reward=547.542! [2023-09-19 10:15:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7236.3, 300 sec: 7008.7). Total num frames: 864256. Throughput: 0: 3501.2, 1: 3501.3. Samples: 616276. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:15:21,682][45164] Avg episode reward: [(0, '582.305'), (1, '388.246')] [2023-09-19 10:15:21,684][45810] Saving new best policy, reward=582.305! [2023-09-19 10:15:21,684][45811] Saving new best policy, reward=388.246! [2023-09-19 10:15:24,149][45899] Updated weights for policy 0, policy_version 1096 (0.0011) [2023-09-19 10:15:24,150][45898] Updated weights for policy 1, policy_version 640 (0.0014) [2023-09-19 10:15:26,681][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 6984.7). Total num frames: 897024. Throughput: 0: 3529.1, 1: 3528.0. Samples: 662886. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:15:26,683][45164] Avg episode reward: [(0, '557.097'), (1, '394.044')] [2023-09-19 10:15:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001104_565248.pth... [2023-09-19 10:15:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000648_331776.pth... [2023-09-19 10:15:26,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000000912_466944.pth [2023-09-19 10:15:26,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000456_233472.pth [2023-09-19 10:15:26,702][45811] Saving new best policy, reward=394.044! [2023-09-19 10:15:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 7045.1). Total num frames: 937984. Throughput: 0: 3450.4, 1: 3450.6. Samples: 703672. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:15:31,682][45164] Avg episode reward: [(0, '599.464'), (1, '410.184')] [2023-09-19 10:15:31,684][45810] Saving new best policy, reward=599.464! [2023-09-19 10:15:31,684][45811] Saving new best policy, reward=410.184! [2023-09-19 10:15:36,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6943.7). Total num frames: 962560. Throughput: 0: 3365.5, 1: 3364.6. Samples: 719636. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:15:36,683][45164] Avg episode reward: [(0, '639.388'), (1, '409.635')] [2023-09-19 10:15:36,684][45810] Saving new best policy, reward=639.388! [2023-09-19 10:15:37,134][45898] Updated weights for policy 1, policy_version 720 (0.0011) [2023-09-19 10:15:37,135][45899] Updated weights for policy 0, policy_version 1176 (0.0014) [2023-09-19 10:15:41,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7000.4). Total num frames: 1003520. Throughput: 0: 3321.7, 1: 3321.9. Samples: 760974. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:15:41,682][45164] Avg episode reward: [(0, '621.038'), (1, '407.944')] [2023-09-19 10:15:41,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001208_618496.pth... [2023-09-19 10:15:41,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000752_385024.pth... [2023-09-19 10:15:41,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000552_282624.pth [2023-09-19 10:15:41,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001008_516096.pth [2023-09-19 10:15:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.6, 300 sec: 6981.0). Total num frames: 1036288. Throughput: 0: 3377.6, 1: 3378.0. Samples: 803068. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:15:46,683][45164] Avg episode reward: [(0, '669.094'), (1, '414.470')] [2023-09-19 10:15:46,685][45810] Saving new best policy, reward=669.094! [2023-09-19 10:15:46,685][45811] Saving new best policy, reward=414.470! [2023-09-19 10:15:48,580][45898] Updated weights for policy 1, policy_version 800 (0.0013) [2023-09-19 10:15:48,580][45899] Updated weights for policy 0, policy_version 1256 (0.0014) [2023-09-19 10:15:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6963.2). Total num frames: 1069056. Throughput: 0: 3420.6, 1: 3421.9. Samples: 824004. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:15:51,682][45164] Avg episode reward: [(0, '665.228'), (1, '439.424')] [2023-09-19 10:15:51,683][45811] Saving new best policy, reward=439.424! [2023-09-19 10:15:56,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6690.1, 300 sec: 6946.8). Total num frames: 1101824. Throughput: 0: 3405.2, 1: 3405.1. Samples: 864692. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:15:56,683][45164] Avg episode reward: [(0, '693.987'), (1, '469.314')] [2023-09-19 10:15:56,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001304_667648.pth... [2023-09-19 10:15:56,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000848_434176.pth... [2023-09-19 10:15:56,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000648_331776.pth [2023-09-19 10:15:56,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001104_565248.pth [2023-09-19 10:15:56,702][45810] Saving new best policy, reward=693.987! [2023-09-19 10:15:56,702][45811] Saving new best policy, reward=469.314! [2023-09-19 10:16:00,316][45899] Updated weights for policy 0, policy_version 1336 (0.0015) [2023-09-19 10:16:00,316][45898] Updated weights for policy 1, policy_version 880 (0.0014) [2023-09-19 10:16:01,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6994.7). Total num frames: 1142784. Throughput: 0: 3468.3, 1: 3468.3. Samples: 908358. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:16:01,682][45164] Avg episode reward: [(0, '683.885'), (1, '462.652')] [2023-09-19 10:16:06,681][45164] Fps is (10 sec: 7373.1, 60 sec: 6826.7, 300 sec: 6978.4). Total num frames: 1175552. Throughput: 0: 3468.3, 1: 3468.0. Samples: 928406. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:16:06,682][45164] Avg episode reward: [(0, '644.650'), (1, '483.113')] [2023-09-19 10:16:06,682][45811] Saving new best policy, reward=483.113! [2023-09-19 10:16:11,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6963.2). Total num frames: 1208320. Throughput: 0: 3423.0, 1: 3424.1. Samples: 971008. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:16:11,683][45164] Avg episode reward: [(0, '745.247'), (1, '519.693')] [2023-09-19 10:16:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000952_487424.pth... [2023-09-19 10:16:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001408_720896.pth... [2023-09-19 10:16:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000752_385024.pth [2023-09-19 10:16:11,700][45811] Saving new best policy, reward=519.693! [2023-09-19 10:16:11,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001208_618496.pth [2023-09-19 10:16:11,703][45810] Saving new best policy, reward=745.247! [2023-09-19 10:16:11,872][45899] Updated weights for policy 0, policy_version 1416 (0.0014) [2023-09-19 10:16:11,872][45898] Updated weights for policy 1, policy_version 960 (0.0013) [2023-09-19 10:16:16,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 7005.6). Total num frames: 1249280. Throughput: 0: 3468.5, 1: 3468.0. Samples: 1015814. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:16:16,682][45164] Avg episode reward: [(0, '706.451'), (1, '562.557')] [2023-09-19 10:16:16,683][45811] Saving new best policy, reward=562.557! [2023-09-19 10:16:21,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6990.5). Total num frames: 1282048. Throughput: 0: 3562.8, 1: 3562.8. Samples: 1040284. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:16:21,682][45164] Avg episode reward: [(0, '754.995'), (1, '579.458')] [2023-09-19 10:16:21,683][45810] Saving new best policy, reward=754.995! [2023-09-19 10:16:21,683][45811] Saving new best policy, reward=579.458! [2023-09-19 10:16:23,120][45899] Updated weights for policy 0, policy_version 1496 (0.0014) [2023-09-19 10:16:23,120][45898] Updated weights for policy 1, policy_version 1040 (0.0014) [2023-09-19 10:16:26,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.8, 300 sec: 7029.3). Total num frames: 1323008. Throughput: 0: 3526.2, 1: 3526.0. Samples: 1078324. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:16:26,682][45164] Avg episode reward: [(0, '795.262'), (1, '594.466')] [2023-09-19 10:16:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001520_778240.pth... [2023-09-19 10:16:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001064_544768.pth... [2023-09-19 10:16:26,695][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001304_667648.pth [2023-09-19 10:16:26,695][45810] Saving new best policy, reward=795.262! [2023-09-19 10:16:26,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000848_434176.pth [2023-09-19 10:16:26,700][45811] Saving new best policy, reward=594.466! [2023-09-19 10:16:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 7014.4). Total num frames: 1355776. Throughput: 0: 3551.2, 1: 3551.2. Samples: 1122674. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:16:31,682][45164] Avg episode reward: [(0, '823.295'), (1, '596.428')] [2023-09-19 10:16:31,683][45810] Saving new best policy, reward=823.295! [2023-09-19 10:16:31,683][45811] Saving new best policy, reward=596.428! [2023-09-19 10:16:34,173][45899] Updated weights for policy 0, policy_version 1576 (0.0014) [2023-09-19 10:16:34,173][45898] Updated weights for policy 1, policy_version 1120 (0.0013) [2023-09-19 10:16:36,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 7050.1). Total num frames: 1396736. Throughput: 0: 3588.2, 1: 3586.9. Samples: 1146884. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:16:36,683][45164] Avg episode reward: [(0, '800.385'), (1, '605.521')] [2023-09-19 10:16:36,684][45811] Saving new best policy, reward=605.521! [2023-09-19 10:16:41,682][45164] Fps is (10 sec: 7372.6, 60 sec: 7099.7, 300 sec: 7035.5). Total num frames: 1429504. Throughput: 0: 3625.6, 1: 3625.4. Samples: 1190988. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:16:41,683][45164] Avg episode reward: [(0, '857.482'), (1, '661.583')] [2023-09-19 10:16:41,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001624_831488.pth... [2023-09-19 10:16:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001168_598016.pth... [2023-09-19 10:16:41,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001408_720896.pth [2023-09-19 10:16:41,698][45810] Saving new best policy, reward=857.482! [2023-09-19 10:16:41,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000000952_487424.pth [2023-09-19 10:16:41,701][45811] Saving new best policy, reward=661.583! [2023-09-19 10:16:45,452][45898] Updated weights for policy 1, policy_version 1200 (0.0013) [2023-09-19 10:16:45,452][45899] Updated weights for policy 0, policy_version 1656 (0.0015) [2023-09-19 10:16:46,682][45164] Fps is (10 sec: 7372.6, 60 sec: 7236.3, 300 sec: 7068.5). Total num frames: 1470464. Throughput: 0: 3625.4, 1: 3625.5. Samples: 1234646. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:16:46,683][45164] Avg episode reward: [(0, '818.375'), (1, '638.405')] [2023-09-19 10:16:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.3, 300 sec: 7054.2). Total num frames: 1503232. Throughput: 0: 3645.2, 1: 3645.3. Samples: 1256482. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:16:51,682][45164] Avg episode reward: [(0, '881.565'), (1, '700.030')] [2023-09-19 10:16:51,684][45810] Saving new best policy, reward=881.565! [2023-09-19 10:16:51,684][45811] Saving new best policy, reward=700.030! [2023-09-19 10:16:56,579][45899] Updated weights for policy 0, policy_version 1736 (0.0015) [2023-09-19 10:16:56,579][45898] Updated weights for policy 1, policy_version 1280 (0.0012) [2023-09-19 10:16:56,682][45164] Fps is (10 sec: 7372.7, 60 sec: 7372.8, 300 sec: 7085.0). Total num frames: 1544192. Throughput: 0: 3654.5, 1: 3654.6. Samples: 1299916. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:16:56,683][45164] Avg episode reward: [(0, '893.837'), (1, '727.886')] [2023-09-19 10:16:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001736_888832.pth... [2023-09-19 10:16:56,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001280_655360.pth... [2023-09-19 10:16:56,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001064_544768.pth [2023-09-19 10:16:56,697][45811] Saving new best policy, reward=727.886! [2023-09-19 10:16:56,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001520_778240.pth [2023-09-19 10:16:56,699][45810] Saving new best policy, reward=893.837! [2023-09-19 10:17:01,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.3, 300 sec: 7071.0). Total num frames: 1576960. Throughput: 0: 3641.5, 1: 3641.6. Samples: 1343554. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:17:01,682][45164] Avg episode reward: [(0, '997.752'), (1, '738.492')] [2023-09-19 10:17:01,683][45810] Saving new best policy, reward=997.752! [2023-09-19 10:17:01,683][45811] Saving new best policy, reward=738.492! [2023-09-19 10:17:06,681][45164] Fps is (10 sec: 6553.9, 60 sec: 7236.3, 300 sec: 7057.7). Total num frames: 1609728. Throughput: 0: 3595.5, 1: 3595.9. Samples: 1363898. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:17:06,683][45164] Avg episode reward: [(0, '960.757'), (1, '746.766')] [2023-09-19 10:17:06,684][45811] Saving new best policy, reward=746.766! [2023-09-19 10:17:09,123][45899] Updated weights for policy 0, policy_version 1816 (0.0012) [2023-09-19 10:17:09,123][45898] Updated weights for policy 1, policy_version 1360 (0.0015) [2023-09-19 10:17:11,682][45164] Fps is (10 sec: 6553.4, 60 sec: 7236.3, 300 sec: 7045.1). Total num frames: 1642496. Throughput: 0: 3584.0, 1: 3582.8. Samples: 1400834. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:17:11,682][45164] Avg episode reward: [(0, '1025.841'), (1, '723.937')] [2023-09-19 10:17:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001832_937984.pth... [2023-09-19 10:17:11,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001376_704512.pth... [2023-09-19 10:17:11,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001624_831488.pth [2023-09-19 10:17:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001168_598016.pth [2023-09-19 10:17:11,701][45810] Saving new best policy, reward=1025.841! [2023-09-19 10:17:16,681][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 7033.1). Total num frames: 1675264. Throughput: 0: 3556.0, 1: 3556.3. Samples: 1442730. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:17:16,683][45164] Avg episode reward: [(0, '964.347'), (1, '726.601')] [2023-09-19 10:17:20,467][45899] Updated weights for policy 0, policy_version 1896 (0.0011) [2023-09-19 10:17:20,469][45898] Updated weights for policy 1, policy_version 1440 (0.0014) [2023-09-19 10:17:21,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.2, 300 sec: 7060.7). Total num frames: 1716224. Throughput: 0: 3534.9, 1: 3536.0. Samples: 1465076. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:17:21,683][45164] Avg episode reward: [(0, '1006.854'), (1, '805.580')] [2023-09-19 10:17:21,684][45811] Saving new best policy, reward=805.580! [2023-09-19 10:17:26,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 7048.9). Total num frames: 1748992. Throughput: 0: 3528.9, 1: 3529.0. Samples: 1508592. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:17:26,682][45164] Avg episode reward: [(0, '985.770'), (1, '803.173')] [2023-09-19 10:17:26,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001936_991232.pth... [2023-09-19 10:17:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001480_757760.pth... [2023-09-19 10:17:26,696][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001736_888832.pth [2023-09-19 10:17:26,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001280_655360.pth [2023-09-19 10:17:31,498][45899] Updated weights for policy 0, policy_version 1976 (0.0011) [2023-09-19 10:17:31,498][45898] Updated weights for policy 1, policy_version 1520 (0.0013) [2023-09-19 10:17:31,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.3, 300 sec: 7074.9). Total num frames: 1789952. Throughput: 0: 3561.8, 1: 3561.6. Samples: 1555194. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:17:31,682][45164] Avg episode reward: [(0, '1054.506'), (1, '769.676')] [2023-09-19 10:17:31,683][45810] Saving new best policy, reward=1054.506! [2023-09-19 10:17:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 7063.3). Total num frames: 1822720. Throughput: 0: 3570.1, 1: 3570.4. Samples: 1577800. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:17:36,682][45164] Avg episode reward: [(0, '1159.032'), (1, '819.512')] [2023-09-19 10:17:36,729][45811] Saving new best policy, reward=819.512! [2023-09-19 10:17:36,730][45810] Saving new best policy, reward=1159.032! [2023-09-19 10:17:41,682][45164] Fps is (10 sec: 7372.6, 60 sec: 7236.3, 300 sec: 7087.9). Total num frames: 1863680. Throughput: 0: 3588.3, 1: 3588.2. Samples: 1622860. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:17:41,683][45164] Avg episode reward: [(0, '1233.852'), (1, '843.819')] [2023-09-19 10:17:41,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002048_1048576.pth... [2023-09-19 10:17:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001592_815104.pth... [2023-09-19 10:17:41,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001832_937984.pth [2023-09-19 10:17:41,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001376_704512.pth [2023-09-19 10:17:41,699][45810] Saving new best policy, reward=1233.852! [2023-09-19 10:17:41,699][45811] Saving new best policy, reward=843.819! [2023-09-19 10:17:42,125][45898] Updated weights for policy 1, policy_version 1600 (0.0014) [2023-09-19 10:17:42,126][45899] Updated weights for policy 0, policy_version 2056 (0.0013) [2023-09-19 10:17:46,681][45164] Fps is (10 sec: 7372.6, 60 sec: 7099.7, 300 sec: 7076.5). Total num frames: 1896448. Throughput: 0: 3597.1, 1: 3598.4. Samples: 1667356. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:17:46,682][45164] Avg episode reward: [(0, '1091.152'), (1, '778.541')] [2023-09-19 10:17:51,681][45164] Fps is (10 sec: 7373.0, 60 sec: 7236.3, 300 sec: 7099.7). Total num frames: 1937408. Throughput: 0: 3630.9, 1: 3631.3. Samples: 1690700. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:17:51,682][45164] Avg episode reward: [(0, '1065.478'), (1, '805.519')] [2023-09-19 10:17:53,446][45898] Updated weights for policy 1, policy_version 1680 (0.0010) [2023-09-19 10:17:53,446][45899] Updated weights for policy 0, policy_version 2136 (0.0013) [2023-09-19 10:17:56,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.8, 300 sec: 7088.6). Total num frames: 1970176. Throughput: 0: 3683.7, 1: 3685.0. Samples: 1732426. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:17:56,682][45164] Avg episode reward: [(0, '1238.077'), (1, '835.667')] [2023-09-19 10:17:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002152_1101824.pth... [2023-09-19 10:17:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001696_868352.pth... [2023-09-19 10:17:56,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000001936_991232.pth [2023-09-19 10:17:56,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001480_757760.pth [2023-09-19 10:17:56,702][45810] Saving new best policy, reward=1238.077! [2023-09-19 10:18:01,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 7110.7). Total num frames: 2011136. Throughput: 0: 3685.4, 1: 3685.5. Samples: 1774420. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:01,682][45164] Avg episode reward: [(0, '1272.836'), (1, '785.668')] [2023-09-19 10:18:01,683][45810] Saving new best policy, reward=1272.836! [2023-09-19 10:18:04,794][45899] Updated weights for policy 0, policy_version 2216 (0.0015) [2023-09-19 10:18:04,794][45898] Updated weights for policy 1, policy_version 1760 (0.0013) [2023-09-19 10:18:06,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.3, 300 sec: 7099.7). Total num frames: 2043904. Throughput: 0: 3689.8, 1: 3690.0. Samples: 1797162. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:06,682][45164] Avg episode reward: [(0, '1293.353'), (1, '786.813')] [2023-09-19 10:18:06,683][45810] Saving new best policy, reward=1293.353! [2023-09-19 10:18:11,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7236.3, 300 sec: 7089.2). Total num frames: 2076672. Throughput: 0: 3674.3, 1: 3674.6. Samples: 1839290. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:11,682][45164] Avg episode reward: [(0, '1326.343'), (1, '735.196')] [2023-09-19 10:18:11,737][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001808_925696.pth... [2023-09-19 10:18:11,739][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002264_1159168.pth... [2023-09-19 10:18:11,741][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001592_815104.pth [2023-09-19 10:18:11,743][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002048_1048576.pth [2023-09-19 10:18:11,743][45810] Saving new best policy, reward=1326.343! [2023-09-19 10:18:16,386][45899] Updated weights for policy 0, policy_version 2296 (0.0014) [2023-09-19 10:18:16,386][45898] Updated weights for policy 1, policy_version 1840 (0.0012) [2023-09-19 10:18:16,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7372.8, 300 sec: 7110.0). Total num frames: 2117632. Throughput: 0: 3650.7, 1: 3650.2. Samples: 1883734. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:18:16,682][45164] Avg episode reward: [(0, '1247.086'), (1, '798.767')] [2023-09-19 10:18:21,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 7099.7). Total num frames: 2150400. Throughput: 0: 3647.3, 1: 3647.2. Samples: 1906054. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:21,682][45164] Avg episode reward: [(0, '1179.210'), (1, '894.458')] [2023-09-19 10:18:21,683][45811] Saving new best policy, reward=894.458! [2023-09-19 10:18:26,682][45164] Fps is (10 sec: 6553.5, 60 sec: 7236.2, 300 sec: 7089.8). Total num frames: 2183168. Throughput: 0: 3613.4, 1: 3613.3. Samples: 1948060. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:26,683][45164] Avg episode reward: [(0, '1206.703'), (1, '935.562')] [2023-09-19 10:18:26,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002360_1208320.pth... [2023-09-19 10:18:26,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001904_974848.pth... [2023-09-19 10:18:26,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001696_868352.pth [2023-09-19 10:18:26,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002152_1101824.pth [2023-09-19 10:18:26,702][45811] Saving new best policy, reward=935.562! [2023-09-19 10:18:27,829][45898] Updated weights for policy 1, policy_version 1920 (0.0013) [2023-09-19 10:18:27,830][45899] Updated weights for policy 0, policy_version 2376 (0.0012) [2023-09-19 10:18:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7236.3, 300 sec: 7109.5). Total num frames: 2224128. Throughput: 0: 3615.8, 1: 3615.6. Samples: 1992766. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:18:31,683][45164] Avg episode reward: [(0, '1149.093'), (1, '940.731')] [2023-09-19 10:18:31,684][45811] Saving new best policy, reward=940.731! [2023-09-19 10:18:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.2, 300 sec: 7099.7). Total num frames: 2256896. Throughput: 0: 3597.6, 1: 3597.7. Samples: 2014492. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:36,683][45164] Avg episode reward: [(0, '1194.948'), (1, '1031.196')] [2023-09-19 10:18:36,684][45811] Saving new best policy, reward=1031.196! [2023-09-19 10:18:39,161][45898] Updated weights for policy 1, policy_version 2000 (0.0013) [2023-09-19 10:18:39,161][45899] Updated weights for policy 0, policy_version 2456 (0.0013) [2023-09-19 10:18:41,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.3, 300 sec: 7118.6). Total num frames: 2297856. Throughput: 0: 3598.0, 1: 3596.8. Samples: 2056190. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:41,682][45164] Avg episode reward: [(0, '1261.981'), (1, '1062.331')] [2023-09-19 10:18:41,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002016_1032192.pth... [2023-09-19 10:18:41,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002472_1265664.pth... [2023-09-19 10:18:41,695][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002264_1159168.pth [2023-09-19 10:18:41,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001808_925696.pth [2023-09-19 10:18:41,697][45811] Saving new best policy, reward=1062.331! [2023-09-19 10:18:46,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.3, 300 sec: 7109.0). Total num frames: 2330624. Throughput: 0: 3607.7, 1: 3607.6. Samples: 2099106. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:46,682][45164] Avg episode reward: [(0, '1398.377'), (1, '1134.930')] [2023-09-19 10:18:46,684][45810] Saving new best policy, reward=1398.377! [2023-09-19 10:18:46,684][45811] Saving new best policy, reward=1134.930! [2023-09-19 10:18:50,779][45898] Updated weights for policy 1, policy_version 2080 (0.0011) [2023-09-19 10:18:50,780][45899] Updated weights for policy 0, policy_version 2536 (0.0015) [2023-09-19 10:18:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7099.7, 300 sec: 7136.8). Total num frames: 2363392. Throughput: 0: 3587.8, 1: 3587.8. Samples: 2120064. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:51,682][45164] Avg episode reward: [(0, '1325.130'), (1, '1022.969')] [2023-09-19 10:18:56,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7099.7, 300 sec: 7136.8). Total num frames: 2396160. Throughput: 0: 3554.0, 1: 3554.0. Samples: 2159146. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:18:56,682][45164] Avg episode reward: [(0, '1287.921'), (1, '934.800')] [2023-09-19 10:18:56,697][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002576_1318912.pth... [2023-09-19 10:18:56,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002360_1208320.pth [2023-09-19 10:18:56,709][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002120_1085440.pth... [2023-09-19 10:18:56,713][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000001904_974848.pth [2023-09-19 10:19:01,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 7164.5). Total num frames: 2437120. Throughput: 0: 3525.4, 1: 3525.9. Samples: 2201040. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:19:01,682][45164] Avg episode reward: [(0, '1244.520'), (1, '953.417')] [2023-09-19 10:19:02,923][45899] Updated weights for policy 0, policy_version 2616 (0.0010) [2023-09-19 10:19:02,923][45898] Updated weights for policy 1, policy_version 2160 (0.0011) [2023-09-19 10:19:06,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 7136.8). Total num frames: 2469888. Throughput: 0: 3506.5, 1: 3506.1. Samples: 2221622. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:19:06,682][45164] Avg episode reward: [(0, '1306.653'), (1, '1023.352')] [2023-09-19 10:19:11,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7099.7, 300 sec: 7136.8). Total num frames: 2502656. Throughput: 0: 3532.8, 1: 3532.8. Samples: 2266016. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:19:11,682][45164] Avg episode reward: [(0, '1266.320'), (1, '985.807')] [2023-09-19 10:19:11,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002672_1368064.pth... [2023-09-19 10:19:11,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002216_1134592.pth... [2023-09-19 10:19:11,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002472_1265664.pth [2023-09-19 10:19:11,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002016_1032192.pth [2023-09-19 10:19:14,434][45899] Updated weights for policy 0, policy_version 2696 (0.0008) [2023-09-19 10:19:14,436][45898] Updated weights for policy 1, policy_version 2240 (0.0015) [2023-09-19 10:19:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7136.8). Total num frames: 2535424. Throughput: 0: 3456.0, 1: 3456.1. Samples: 2303812. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:19:16,682][45164] Avg episode reward: [(0, '1303.995'), (1, '1004.926')] [2023-09-19 10:19:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 7109.0). Total num frames: 2568192. Throughput: 0: 3413.0, 1: 3412.8. Samples: 2321652. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:19:21,683][45164] Avg episode reward: [(0, '1349.787'), (1, '1018.519')] [2023-09-19 10:19:26,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 2600960. Throughput: 0: 3398.4, 1: 3399.7. Samples: 2362106. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:19:26,682][45164] Avg episode reward: [(0, '1334.560'), (1, '1028.041')] [2023-09-19 10:19:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002768_1417216.pth... [2023-09-19 10:19:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002312_1183744.pth... [2023-09-19 10:19:26,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002576_1318912.pth [2023-09-19 10:19:26,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002120_1085440.pth [2023-09-19 10:19:26,968][45898] Updated weights for policy 1, policy_version 2320 (0.0015) [2023-09-19 10:19:26,969][45899] Updated weights for policy 0, policy_version 2776 (0.0016) [2023-09-19 10:19:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 7109.0). Total num frames: 2641920. Throughput: 0: 3433.4, 1: 3433.1. Samples: 2408102. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:19:31,683][45164] Avg episode reward: [(0, '1474.155'), (1, '1073.856')] [2023-09-19 10:19:31,684][45810] Saving new best policy, reward=1474.155! [2023-09-19 10:19:36,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 2674688. Throughput: 0: 3410.1, 1: 3410.1. Samples: 2426972. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:19:36,683][45164] Avg episode reward: [(0, '1411.132'), (1, '1162.051')] [2023-09-19 10:19:36,684][45811] Saving new best policy, reward=1162.051! [2023-09-19 10:19:38,480][45899] Updated weights for policy 0, policy_version 2856 (0.0014) [2023-09-19 10:19:38,480][45898] Updated weights for policy 1, policy_version 2400 (0.0015) [2023-09-19 10:19:41,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 7053.5). Total num frames: 2707456. Throughput: 0: 3453.0, 1: 3453.0. Samples: 2469914. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:19:41,682][45164] Avg episode reward: [(0, '1410.873'), (1, '1086.158')] [2023-09-19 10:19:41,688][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002872_1470464.pth... [2023-09-19 10:19:41,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002416_1236992.pth... [2023-09-19 10:19:41,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002216_1134592.pth [2023-09-19 10:19:41,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002672_1368064.pth [2023-09-19 10:19:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 7053.4). Total num frames: 2748416. Throughput: 0: 3493.6, 1: 3493.6. Samples: 2515466. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:19:46,683][45164] Avg episode reward: [(0, '1412.771'), (1, '1103.708')] [2023-09-19 10:19:49,657][45898] Updated weights for policy 1, policy_version 2480 (0.0013) [2023-09-19 10:19:49,658][45899] Updated weights for policy 0, policy_version 2936 (0.0013) [2023-09-19 10:19:51,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 7053.4). Total num frames: 2781184. Throughput: 0: 3489.5, 1: 3489.9. Samples: 2535692. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:19:51,682][45164] Avg episode reward: [(0, '1421.836'), (1, '1172.660')] [2023-09-19 10:19:51,684][45811] Saving new best policy, reward=1172.660! [2023-09-19 10:19:56,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7053.4). Total num frames: 2813952. Throughput: 0: 3431.4, 1: 3431.5. Samples: 2574844. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:19:56,683][45164] Avg episode reward: [(0, '1539.459'), (1, '1277.867')] [2023-09-19 10:19:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002520_1290240.pth... [2023-09-19 10:19:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002976_1523712.pth... [2023-09-19 10:19:56,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002312_1183744.pth [2023-09-19 10:19:56,698][45811] Saving new best policy, reward=1277.867! [2023-09-19 10:19:56,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002768_1417216.pth [2023-09-19 10:19:56,699][45810] Saving new best policy, reward=1539.459! [2023-09-19 10:20:01,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 7053.4). Total num frames: 2846720. Throughput: 0: 3480.5, 1: 3480.5. Samples: 2617056. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:20:01,682][45164] Avg episode reward: [(0, '1945.644'), (1, '1391.114')] [2023-09-19 10:20:01,708][45811] Saving new best policy, reward=1391.114! [2023-09-19 10:20:01,718][45810] Saving new best policy, reward=1945.644! [2023-09-19 10:20:01,720][45898] Updated weights for policy 1, policy_version 2560 (0.0013) [2023-09-19 10:20:01,720][45899] Updated weights for policy 0, policy_version 3016 (0.0015) [2023-09-19 10:20:06,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 2887680. Throughput: 0: 3541.5, 1: 3541.6. Samples: 2640392. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:20:06,683][45164] Avg episode reward: [(0, '2400.422'), (1, '1417.756')] [2023-09-19 10:20:06,684][45810] Saving new best policy, reward=2400.422! [2023-09-19 10:20:06,684][45811] Saving new best policy, reward=1417.756! [2023-09-19 10:20:11,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 2920448. Throughput: 0: 3552.0, 1: 3551.9. Samples: 2681778. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:20:11,682][45164] Avg episode reward: [(0, '2606.517'), (1, '1481.005')] [2023-09-19 10:20:11,688][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002624_1343488.pth... [2023-09-19 10:20:11,688][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003080_1576960.pth... [2023-09-19 10:20:11,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002416_1236992.pth [2023-09-19 10:20:11,696][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002872_1470464.pth [2023-09-19 10:20:11,697][45811] Saving new best policy, reward=1481.005! [2023-09-19 10:20:11,697][45810] Saving new best policy, reward=2606.517! [2023-09-19 10:20:13,907][45898] Updated weights for policy 1, policy_version 2640 (0.0013) [2023-09-19 10:20:13,907][45899] Updated weights for policy 0, policy_version 3096 (0.0014) [2023-09-19 10:20:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 2953216. Throughput: 0: 3462.6, 1: 3462.0. Samples: 2719712. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:20:16,683][45164] Avg episode reward: [(0, '2067.628'), (1, '1442.669')] [2023-09-19 10:20:21,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 7109.0). Total num frames: 2994176. Throughput: 0: 3515.4, 1: 3515.4. Samples: 2743358. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:20:21,682][45164] Avg episode reward: [(0, '1726.389'), (1, '1477.082')] [2023-09-19 10:20:25,416][45898] Updated weights for policy 1, policy_version 2720 (0.0013) [2023-09-19 10:20:25,417][45899] Updated weights for policy 0, policy_version 3176 (0.0015) [2023-09-19 10:20:26,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 7081.2). Total num frames: 3026944. Throughput: 0: 3498.6, 1: 3497.7. Samples: 2784748. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:20:26,682][45164] Avg episode reward: [(0, '1774.692'), (1, '1319.508')] [2023-09-19 10:20:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003184_1630208.pth... [2023-09-19 10:20:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002728_1396736.pth... [2023-09-19 10:20:26,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002520_1290240.pth [2023-09-19 10:20:26,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000002976_1523712.pth [2023-09-19 10:20:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.8, 300 sec: 7136.8). Total num frames: 3067904. Throughput: 0: 3508.7, 1: 3508.7. Samples: 2831246. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:20:31,682][45164] Avg episode reward: [(0, '1643.976'), (1, '1235.925')] [2023-09-19 10:20:36,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 3092480. Throughput: 0: 3502.1, 1: 3500.7. Samples: 2850816. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:20:36,683][45164] Avg episode reward: [(0, '1563.493'), (1, '1276.928')] [2023-09-19 10:20:36,713][45898] Updated weights for policy 1, policy_version 2800 (0.0012) [2023-09-19 10:20:36,713][45899] Updated weights for policy 0, policy_version 3256 (0.0014) [2023-09-19 10:20:41,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 3125248. Throughput: 0: 3520.4, 1: 3519.2. Samples: 2891624. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:20:41,682][45164] Avg episode reward: [(0, '1633.388'), (1, '1405.152')] [2023-09-19 10:20:41,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003280_1679360.pth... [2023-09-19 10:20:41,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002824_1445888.pth... [2023-09-19 10:20:41,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003080_1576960.pth [2023-09-19 10:20:41,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002624_1343488.pth [2023-09-19 10:20:46,681][45164] Fps is (10 sec: 7373.0, 60 sec: 6963.2, 300 sec: 7109.0). Total num frames: 3166208. Throughput: 0: 3473.6, 1: 3473.6. Samples: 2929678. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:20:46,682][45164] Avg episode reward: [(0, '1612.558'), (1, '1435.661')] [2023-09-19 10:20:48,781][45899] Updated weights for policy 0, policy_version 3336 (0.0014) [2023-09-19 10:20:48,782][45898] Updated weights for policy 1, policy_version 2880 (0.0014) [2023-09-19 10:20:51,681][45164] Fps is (10 sec: 7372.6, 60 sec: 6963.2, 300 sec: 7109.0). Total num frames: 3198976. Throughput: 0: 3464.0, 1: 3463.9. Samples: 2952146. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:20:51,683][45164] Avg episode reward: [(0, '1482.863'), (1, '1390.821')] [2023-09-19 10:20:56,682][45164] Fps is (10 sec: 6553.4, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 3231744. Throughput: 0: 3448.8, 1: 3448.7. Samples: 2992170. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:20:56,682][45164] Avg episode reward: [(0, '1611.378'), (1, '1619.852')] [2023-09-19 10:20:56,688][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002928_1499136.pth... [2023-09-19 10:20:56,688][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003384_1732608.pth... [2023-09-19 10:20:56,692][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002728_1396736.pth [2023-09-19 10:20:56,692][45811] Saving new best policy, reward=1619.852! [2023-09-19 10:20:56,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003184_1630208.pth [2023-09-19 10:21:00,948][45898] Updated weights for policy 1, policy_version 2960 (0.0015) [2023-09-19 10:21:00,948][45899] Updated weights for policy 0, policy_version 3416 (0.0014) [2023-09-19 10:21:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 3264512. Throughput: 0: 3474.4, 1: 3475.3. Samples: 3032448. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:21:01,683][45164] Avg episode reward: [(0, '1905.453'), (1, '1720.455')] [2023-09-19 10:21:01,684][45811] Saving new best policy, reward=1720.455! [2023-09-19 10:21:06,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 7081.2). Total num frames: 3297280. Throughput: 0: 3422.0, 1: 3421.6. Samples: 3051324. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:21:06,683][45164] Avg episode reward: [(0, '1960.793'), (1, '1658.883')] [2023-09-19 10:21:11,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 7053.5). Total num frames: 3330048. Throughput: 0: 3431.1, 1: 3431.9. Samples: 3093582. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:21:11,682][45164] Avg episode reward: [(0, '1844.327'), (1, '1390.364')] [2023-09-19 10:21:11,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003480_1781760.pth... [2023-09-19 10:21:11,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003024_1548288.pth... [2023-09-19 10:21:11,693][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003280_1679360.pth [2023-09-19 10:21:11,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002824_1445888.pth [2023-09-19 10:21:13,063][45898] Updated weights for policy 1, policy_version 3040 (0.0011) [2023-09-19 10:21:13,063][45899] Updated weights for policy 0, policy_version 3496 (0.0014) [2023-09-19 10:21:16,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 7081.2). Total num frames: 3371008. Throughput: 0: 3403.2, 1: 3402.4. Samples: 3137500. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:21:16,682][45164] Avg episode reward: [(0, '1668.474'), (1, '1324.030')] [2023-09-19 10:21:21,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 7053.4). Total num frames: 3403776. Throughput: 0: 3408.8, 1: 3410.0. Samples: 3157664. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:21:21,683][45164] Avg episode reward: [(0, '1616.032'), (1, '1234.587')] [2023-09-19 10:21:24,198][45899] Updated weights for policy 0, policy_version 3576 (0.0014) [2023-09-19 10:21:24,198][45898] Updated weights for policy 1, policy_version 3120 (0.0014) [2023-09-19 10:21:26,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.6, 300 sec: 7053.4). Total num frames: 3436544. Throughput: 0: 3459.7, 1: 3459.8. Samples: 3203004. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:21:26,682][45164] Avg episode reward: [(0, '1876.417'), (1, '1241.384')] [2023-09-19 10:21:26,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003584_1835008.pth... [2023-09-19 10:21:26,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003384_1732608.pth [2023-09-19 10:21:26,743][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003136_1605632.pth... [2023-09-19 10:21:26,750][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000002928_1499136.pth [2023-09-19 10:21:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 7053.4). Total num frames: 3477504. Throughput: 0: 3508.2, 1: 3507.9. Samples: 3245406. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:21:31,683][45164] Avg episode reward: [(0, '1971.593'), (1, '1217.752')] [2023-09-19 10:21:35,704][45899] Updated weights for policy 0, policy_version 3656 (0.0011) [2023-09-19 10:21:35,704][45898] Updated weights for policy 1, policy_version 3200 (0.0013) [2023-09-19 10:21:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 7053.5). Total num frames: 3510272. Throughput: 0: 3501.1, 1: 3500.0. Samples: 3267198. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:21:36,682][45164] Avg episode reward: [(0, '2098.494'), (1, '1328.220')] [2023-09-19 10:21:41,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 7053.5). Total num frames: 3551232. Throughput: 0: 3539.2, 1: 3539.3. Samples: 3310706. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:21:41,683][45164] Avg episode reward: [(0, '2216.982'), (1, '1438.596')] [2023-09-19 10:21:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003696_1892352.pth... [2023-09-19 10:21:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003240_1658880.pth... [2023-09-19 10:21:41,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003480_1781760.pth [2023-09-19 10:21:41,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003024_1548288.pth [2023-09-19 10:21:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 7053.4). Total num frames: 3584000. Throughput: 0: 3543.3, 1: 3543.2. Samples: 3351338. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:21:46,683][45164] Avg episode reward: [(0, '2406.868'), (1, '1554.587')] [2023-09-19 10:21:47,062][45899] Updated weights for policy 0, policy_version 3736 (0.0016) [2023-09-19 10:21:47,062][45898] Updated weights for policy 1, policy_version 3280 (0.0016) [2023-09-19 10:21:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7025.7). Total num frames: 3616768. Throughput: 0: 3540.3, 1: 3540.2. Samples: 3369944. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:21:51,683][45164] Avg episode reward: [(0, '2296.695'), (1, '1703.507')] [2023-09-19 10:21:56,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7025.7). Total num frames: 3649536. Throughput: 0: 3507.3, 1: 3507.4. Samples: 3409244. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:21:56,682][45164] Avg episode reward: [(0, '2116.386'), (1, '1586.069')] [2023-09-19 10:21:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003792_1941504.pth... [2023-09-19 10:21:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003336_1708032.pth... [2023-09-19 10:21:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003136_1605632.pth [2023-09-19 10:21:56,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003584_1835008.pth [2023-09-19 10:21:59,492][45899] Updated weights for policy 0, policy_version 3816 (0.0013) [2023-09-19 10:21:59,493][45898] Updated weights for policy 1, policy_version 3360 (0.0013) [2023-09-19 10:22:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 7025.7). Total num frames: 3682304. Throughput: 0: 3483.9, 1: 3484.4. Samples: 3451076. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:22:01,682][45164] Avg episode reward: [(0, '2081.634'), (1, '1468.189')] [2023-09-19 10:22:06,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 7025.7). Total num frames: 3715072. Throughput: 0: 3451.6, 1: 3451.4. Samples: 3468300. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:22:06,683][45164] Avg episode reward: [(0, '1927.798'), (1, '1544.452')] [2023-09-19 10:22:11,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 7025.7). Total num frames: 3747840. Throughput: 0: 3419.2, 1: 3420.2. Samples: 3510778. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:22:11,682][45164] Avg episode reward: [(0, '2118.889'), (1, '1483.726')] [2023-09-19 10:22:11,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003888_1990656.pth... [2023-09-19 10:22:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003432_1757184.pth... [2023-09-19 10:22:11,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003240_1658880.pth [2023-09-19 10:22:11,705][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003696_1892352.pth [2023-09-19 10:22:11,932][45899] Updated weights for policy 0, policy_version 3896 (0.0012) [2023-09-19 10:22:11,932][45898] Updated weights for policy 1, policy_version 3440 (0.0012) [2023-09-19 10:22:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6997.9). Total num frames: 3780608. Throughput: 0: 3359.1, 1: 3359.1. Samples: 3547724. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:22:16,682][45164] Avg episode reward: [(0, '2074.831'), (1, '1562.637')] [2023-09-19 10:22:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6997.9). Total num frames: 3813376. Throughput: 0: 3362.5, 1: 3363.4. Samples: 3569864. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:22:21,682][45164] Avg episode reward: [(0, '2103.350'), (1, '1739.682')] [2023-09-19 10:22:21,684][45811] Saving new best policy, reward=1739.682! [2023-09-19 10:22:24,272][45898] Updated weights for policy 1, policy_version 3520 (0.0015) [2023-09-19 10:22:24,272][45899] Updated weights for policy 0, policy_version 3976 (0.0016) [2023-09-19 10:22:26,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 3854336. Throughput: 0: 3341.3, 1: 3341.6. Samples: 3611434. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:22:26,682][45164] Avg episode reward: [(0, '2372.791'), (1, '2027.821')] [2023-09-19 10:22:26,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003992_2043904.pth... [2023-09-19 10:22:26,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003536_1810432.pth... [2023-09-19 10:22:26,695][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003792_1941504.pth [2023-09-19 10:22:26,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003336_1708032.pth [2023-09-19 10:22:26,696][45811] Saving new best policy, reward=2027.821! [2023-09-19 10:22:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 6997.9). Total num frames: 3887104. Throughput: 0: 3381.3, 1: 3381.2. Samples: 3655650. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:22:31,682][45164] Avg episode reward: [(0, '2791.350'), (1, '2137.725')] [2023-09-19 10:22:31,684][45810] Saving new best policy, reward=2791.350! [2023-09-19 10:22:31,684][45811] Saving new best policy, reward=2137.725! [2023-09-19 10:22:35,808][45899] Updated weights for policy 0, policy_version 4056 (0.0016) [2023-09-19 10:22:35,809][45898] Updated weights for policy 1, policy_version 3600 (0.0015) [2023-09-19 10:22:36,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 3919872. Throughput: 0: 3397.8, 1: 3398.0. Samples: 3675756. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:22:36,682][45164] Avg episode reward: [(0, '3050.314'), (1, '2084.162')] [2023-09-19 10:22:36,684][45810] Saving new best policy, reward=3050.314! [2023-09-19 10:22:41,682][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6970.1). Total num frames: 3952640. Throughput: 0: 3407.1, 1: 3407.0. Samples: 3715882. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:22:41,683][45164] Avg episode reward: [(0, '2771.731'), (1, '2248.865')] [2023-09-19 10:22:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004088_2093056.pth... [2023-09-19 10:22:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003632_1859584.pth... [2023-09-19 10:22:41,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003432_1757184.pth [2023-09-19 10:22:41,701][45811] Saving new best policy, reward=2248.865! [2023-09-19 10:22:41,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003888_1990656.pth [2023-09-19 10:22:46,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6690.1, 300 sec: 6942.4). Total num frames: 3985408. Throughput: 0: 3384.6, 1: 3384.9. Samples: 3755702. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:22:46,683][45164] Avg episode reward: [(0, '2773.491'), (1, '2206.710')] [2023-09-19 10:22:48,001][45898] Updated weights for policy 1, policy_version 3680 (0.0014) [2023-09-19 10:22:48,001][45899] Updated weights for policy 0, policy_version 4136 (0.0014) [2023-09-19 10:22:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 4026368. Throughput: 0: 3425.1, 1: 3424.0. Samples: 3776512. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:22:51,682][45164] Avg episode reward: [(0, '2886.836'), (1, '2197.121')] [2023-09-19 10:22:56,682][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 4059136. Throughput: 0: 3397.1, 1: 3397.1. Samples: 3816516. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:22:56,682][45164] Avg episode reward: [(0, '3122.926'), (1, '1973.234')] [2023-09-19 10:22:56,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004192_2146304.pth... [2023-09-19 10:22:56,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003736_1912832.pth... [2023-09-19 10:22:56,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003536_1810432.pth [2023-09-19 10:22:56,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000003992_2043904.pth [2023-09-19 10:22:56,699][45810] Saving new best policy, reward=3122.926! [2023-09-19 10:22:59,524][45898] Updated weights for policy 1, policy_version 3760 (0.0015) [2023-09-19 10:22:59,524][45899] Updated weights for policy 0, policy_version 4216 (0.0015) [2023-09-19 10:23:01,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 4091904. Throughput: 0: 3475.9, 1: 3476.1. Samples: 3860562. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:23:01,682][45164] Avg episode reward: [(0, '3214.176'), (1, '1934.216')] [2023-09-19 10:23:01,683][45810] Saving new best policy, reward=3214.176! [2023-09-19 10:23:06,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 4124672. Throughput: 0: 3444.2, 1: 3444.1. Samples: 3879838. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:23:06,682][45164] Avg episode reward: [(0, '3485.222'), (1, '2008.618')] [2023-09-19 10:23:06,684][45810] Saving new best policy, reward=3485.222! [2023-09-19 10:23:11,222][45898] Updated weights for policy 1, policy_version 3840 (0.0016) [2023-09-19 10:23:11,222][45899] Updated weights for policy 0, policy_version 4296 (0.0016) [2023-09-19 10:23:11,682][45164] Fps is (10 sec: 7372.6, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 4165632. Throughput: 0: 3473.3, 1: 3471.9. Samples: 3923968. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:23:11,682][45164] Avg episode reward: [(0, '3433.934'), (1, '2060.905')] [2023-09-19 10:23:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003840_1966080.pth... [2023-09-19 10:23:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004296_2199552.pth... [2023-09-19 10:23:11,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003632_1859584.pth [2023-09-19 10:23:11,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004088_2093056.pth [2023-09-19 10:23:16,681][45164] Fps is (10 sec: 8192.0, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 4206592. Throughput: 0: 3485.9, 1: 3486.2. Samples: 3969396. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:23:16,682][45164] Avg episode reward: [(0, '3263.205'), (1, '2035.645')] [2023-09-19 10:23:21,681][45164] Fps is (10 sec: 7373.0, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 4239360. Throughput: 0: 3531.7, 1: 3531.8. Samples: 3993616. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:23:21,682][45164] Avg episode reward: [(0, '3097.760'), (1, '2103.144')] [2023-09-19 10:23:21,988][45899] Updated weights for policy 0, policy_version 4376 (0.0016) [2023-09-19 10:23:21,988][45898] Updated weights for policy 1, policy_version 3920 (0.0014) [2023-09-19 10:23:26,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.8, 300 sec: 6970.1). Total num frames: 4280320. Throughput: 0: 3571.8, 1: 3571.7. Samples: 4037340. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:23:26,682][45164] Avg episode reward: [(0, '3101.796'), (1, '2163.264')] [2023-09-19 10:23:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003952_2023424.pth... [2023-09-19 10:23:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004408_2256896.pth... [2023-09-19 10:23:26,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003736_1912832.pth [2023-09-19 10:23:26,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004192_2146304.pth [2023-09-19 10:23:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 4313088. Throughput: 0: 3620.1, 1: 3619.7. Samples: 4081492. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:23:31,682][45164] Avg episode reward: [(0, '3257.721'), (1, '2132.040')] [2023-09-19 10:23:33,215][45899] Updated weights for policy 0, policy_version 4456 (0.0013) [2023-09-19 10:23:33,215][45898] Updated weights for policy 1, policy_version 4000 (0.0014) [2023-09-19 10:23:36,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 6970.1). Total num frames: 4354048. Throughput: 0: 3635.1, 1: 3636.3. Samples: 4103722. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:23:36,682][45164] Avg episode reward: [(0, '3027.391'), (1, '2107.167')] [2023-09-19 10:23:41,681][45164] Fps is (10 sec: 8192.1, 60 sec: 7372.8, 300 sec: 6997.9). Total num frames: 4395008. Throughput: 0: 3716.1, 1: 3716.1. Samples: 4150966. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:23:41,682][45164] Avg episode reward: [(0, '2732.403'), (1, '2102.375')] [2023-09-19 10:23:41,687][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004520_2314240.pth... [2023-09-19 10:23:41,687][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004064_2080768.pth... [2023-09-19 10:23:41,692][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004296_2199552.pth [2023-09-19 10:23:41,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003840_1966080.pth [2023-09-19 10:23:43,981][45899] Updated weights for policy 0, policy_version 4536 (0.0013) [2023-09-19 10:23:43,981][45898] Updated weights for policy 1, policy_version 4080 (0.0012) [2023-09-19 10:23:46,681][45164] Fps is (10 sec: 6553.5, 60 sec: 7236.3, 300 sec: 6970.1). Total num frames: 4419584. Throughput: 0: 3670.3, 1: 3669.9. Samples: 4190874. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:23:46,683][45164] Avg episode reward: [(0, '2523.101'), (1, '2199.488')] [2023-09-19 10:23:51,682][45164] Fps is (10 sec: 5733.8, 60 sec: 7099.6, 300 sec: 6970.1). Total num frames: 4452352. Throughput: 0: 3644.6, 1: 3644.4. Samples: 4207848. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:23:51,683][45164] Avg episode reward: [(0, '2711.533'), (1, '2294.201')] [2023-09-19 10:23:51,684][45811] Saving new best policy, reward=2294.201! [2023-09-19 10:23:56,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 4485120. Throughput: 0: 3584.2, 1: 3585.6. Samples: 4246608. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:23:56,682][45164] Avg episode reward: [(0, '2631.807'), (1, '2053.155')] [2023-09-19 10:23:56,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004608_2359296.pth... [2023-09-19 10:23:56,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004152_2125824.pth... [2023-09-19 10:23:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000003952_2023424.pth [2023-09-19 10:23:56,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004408_2256896.pth [2023-09-19 10:23:56,897][45898] Updated weights for policy 1, policy_version 4160 (0.0015) [2023-09-19 10:23:56,897][45899] Updated weights for policy 0, policy_version 4616 (0.0013) [2023-09-19 10:24:01,681][45164] Fps is (10 sec: 7373.5, 60 sec: 7236.3, 300 sec: 6970.1). Total num frames: 4526080. Throughput: 0: 3591.9, 1: 3590.7. Samples: 4292610. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:24:01,682][45164] Avg episode reward: [(0, '2578.228'), (1, '1818.352')] [2023-09-19 10:24:06,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 6970.1). Total num frames: 4558848. Throughput: 0: 3563.2, 1: 3563.2. Samples: 4314304. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:24:06,683][45164] Avg episode reward: [(0, '2731.465'), (1, '1933.648')] [2023-09-19 10:24:08,586][45898] Updated weights for policy 1, policy_version 4240 (0.0016) [2023-09-19 10:24:08,586][45899] Updated weights for policy 0, policy_version 4696 (0.0015) [2023-09-19 10:24:11,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7099.8, 300 sec: 6970.1). Total num frames: 4591616. Throughput: 0: 3480.5, 1: 3480.5. Samples: 4350586. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:24:11,682][45164] Avg episode reward: [(0, '2809.368'), (1, '1974.882')] [2023-09-19 10:24:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004256_2179072.pth... [2023-09-19 10:24:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004712_2412544.pth... [2023-09-19 10:24:11,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004064_2080768.pth [2023-09-19 10:24:11,705][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004520_2314240.pth [2023-09-19 10:24:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 4624384. Throughput: 0: 3455.3, 1: 3455.7. Samples: 4392488. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:24:16,682][45164] Avg episode reward: [(0, '3059.808'), (1, '1914.647')] [2023-09-19 10:24:20,596][45899] Updated weights for policy 0, policy_version 4776 (0.0014) [2023-09-19 10:24:20,596][45898] Updated weights for policy 1, policy_version 4320 (0.0014) [2023-09-19 10:24:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 4657152. Throughput: 0: 3450.7, 1: 3450.7. Samples: 4414282. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:24:21,682][45164] Avg episode reward: [(0, '3141.452'), (1, '2034.220')] [2023-09-19 10:24:26,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 4698112. Throughput: 0: 3376.0, 1: 3376.0. Samples: 4454808. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:24:26,682][45164] Avg episode reward: [(0, '3297.817'), (1, '2038.720')] [2023-09-19 10:24:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004816_2465792.pth... [2023-09-19 10:24:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004360_2232320.pth... [2023-09-19 10:24:26,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004608_2359296.pth [2023-09-19 10:24:26,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004152_2125824.pth [2023-09-19 10:24:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 4730880. Throughput: 0: 3405.9, 1: 3405.2. Samples: 4497374. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:24:31,682][45164] Avg episode reward: [(0, '3419.584'), (1, '2125.725')] [2023-09-19 10:24:32,516][45898] Updated weights for policy 1, policy_version 4400 (0.0011) [2023-09-19 10:24:32,516][45899] Updated weights for policy 0, policy_version 4856 (0.0013) [2023-09-19 10:24:36,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 4763648. Throughput: 0: 3448.2, 1: 3448.6. Samples: 4518194. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:24:36,682][45164] Avg episode reward: [(0, '3511.622'), (1, '2143.259')] [2023-09-19 10:24:36,683][45810] Saving new best policy, reward=3511.622! [2023-09-19 10:24:41,682][45164] Fps is (10 sec: 6553.4, 60 sec: 6690.1, 300 sec: 6942.4). Total num frames: 4796416. Throughput: 0: 3471.1, 1: 3470.7. Samples: 4558990. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:24:41,682][45164] Avg episode reward: [(0, '3415.602'), (1, '1879.173')] [2023-09-19 10:24:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004912_2514944.pth... [2023-09-19 10:24:41,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004456_2281472.pth... [2023-09-19 10:24:41,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004712_2412544.pth [2023-09-19 10:24:41,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004256_2179072.pth [2023-09-19 10:24:44,614][45899] Updated weights for policy 0, policy_version 4936 (0.0013) [2023-09-19 10:24:44,615][45898] Updated weights for policy 1, policy_version 4480 (0.0012) [2023-09-19 10:24:46,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 4829184. Throughput: 0: 3413.7, 1: 3414.9. Samples: 4599898. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:24:46,682][45164] Avg episode reward: [(0, '3187.425'), (1, '2042.614')] [2023-09-19 10:24:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.3, 300 sec: 6970.1). Total num frames: 4870144. Throughput: 0: 3439.2, 1: 3439.4. Samples: 4623842. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:24:51,683][45164] Avg episode reward: [(0, '3127.572'), (1, '2107.390')] [2023-09-19 10:24:55,384][45899] Updated weights for policy 0, policy_version 5016 (0.0012) [2023-09-19 10:24:55,384][45898] Updated weights for policy 1, policy_version 4560 (0.0014) [2023-09-19 10:24:56,681][45164] Fps is (10 sec: 8191.9, 60 sec: 7099.7, 300 sec: 6997.9). Total num frames: 4911104. Throughput: 0: 3529.0, 1: 3529.2. Samples: 4668204. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:24:56,682][45164] Avg episode reward: [(0, '3399.772'), (1, '2021.558')] [2023-09-19 10:24:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004568_2338816.pth... [2023-09-19 10:24:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005024_2572288.pth... [2023-09-19 10:24:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004360_2232320.pth [2023-09-19 10:24:56,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004816_2465792.pth [2023-09-19 10:25:01,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 4943872. Throughput: 0: 3549.9, 1: 3549.7. Samples: 4711970. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:25:01,682][45164] Avg episode reward: [(0, '3773.431'), (1, '2116.065')] [2023-09-19 10:25:01,683][45810] Saving new best policy, reward=3773.431! [2023-09-19 10:25:06,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 4976640. Throughput: 0: 3547.0, 1: 3547.2. Samples: 4733524. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:25:06,683][45164] Avg episode reward: [(0, '4064.102'), (1, '2242.007')] [2023-09-19 10:25:06,710][45810] Saving new best policy, reward=4064.102! [2023-09-19 10:25:06,722][45898] Updated weights for policy 1, policy_version 4640 (0.0013) [2023-09-19 10:25:06,723][45899] Updated weights for policy 0, policy_version 5096 (0.0014) [2023-09-19 10:25:11,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6997.9). Total num frames: 5017600. Throughput: 0: 3568.0, 1: 3566.8. Samples: 4775876. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:25:11,683][45164] Avg episode reward: [(0, '4293.432'), (1, '2185.890')] [2023-09-19 10:25:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005128_2625536.pth... [2023-09-19 10:25:11,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004672_2392064.pth... [2023-09-19 10:25:11,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000004912_2514944.pth [2023-09-19 10:25:11,699][45810] Saving new best policy, reward=4293.432! [2023-09-19 10:25:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004456_2281472.pth [2023-09-19 10:25:16,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 5050368. Throughput: 0: 3537.3, 1: 3538.4. Samples: 4815784. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:25:16,682][45164] Avg episode reward: [(0, '4333.449'), (1, '2306.995')] [2023-09-19 10:25:16,684][45810] Saving new best policy, reward=4333.449! [2023-09-19 10:25:16,684][45811] Saving new best policy, reward=2306.995! [2023-09-19 10:25:19,171][45898] Updated weights for policy 1, policy_version 4720 (0.0013) [2023-09-19 10:25:19,172][45899] Updated weights for policy 0, policy_version 5176 (0.0013) [2023-09-19 10:25:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 5083136. Throughput: 0: 3501.7, 1: 3500.3. Samples: 4833284. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:25:21,682][45164] Avg episode reward: [(0, '4169.945'), (1, '2329.035')] [2023-09-19 10:25:21,683][45811] Saving new best policy, reward=2329.035! [2023-09-19 10:25:26,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 5115904. Throughput: 0: 3520.3, 1: 3520.5. Samples: 4875828. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:25:26,683][45164] Avg episode reward: [(0, '4195.541'), (1, '2600.305')] [2023-09-19 10:25:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005224_2674688.pth... [2023-09-19 10:25:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004768_2441216.pth... [2023-09-19 10:25:26,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004568_2338816.pth [2023-09-19 10:25:26,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005024_2572288.pth [2023-09-19 10:25:26,698][45811] Saving new best policy, reward=2600.305! [2023-09-19 10:25:30,830][45899] Updated weights for policy 0, policy_version 5256 (0.0013) [2023-09-19 10:25:30,830][45898] Updated weights for policy 1, policy_version 4800 (0.0015) [2023-09-19 10:25:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 5148672. Throughput: 0: 3528.8, 1: 3528.6. Samples: 4917478. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:25:31,682][45164] Avg episode reward: [(0, '4218.679'), (1, '2728.307')] [2023-09-19 10:25:31,683][45811] Saving new best policy, reward=2728.307! [2023-09-19 10:25:36,681][45164] Fps is (10 sec: 5734.5, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 5173248. Throughput: 0: 3436.4, 1: 3436.2. Samples: 4933110. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:25:36,682][45164] Avg episode reward: [(0, '4097.744'), (1, '2692.961')] [2023-09-19 10:25:41,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 5214208. Throughput: 0: 3381.3, 1: 3380.2. Samples: 4972472. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:25:41,682][45164] Avg episode reward: [(0, '4270.629'), (1, '2681.734')] [2023-09-19 10:25:41,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005320_2723840.pth... [2023-09-19 10:25:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004864_2490368.pth... [2023-09-19 10:25:41,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004672_2392064.pth [2023-09-19 10:25:41,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005128_2625536.pth [2023-09-19 10:25:43,581][45899] Updated weights for policy 0, policy_version 5336 (0.0013) [2023-09-19 10:25:43,582][45898] Updated weights for policy 1, policy_version 4880 (0.0011) [2023-09-19 10:25:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 5246976. Throughput: 0: 3375.2, 1: 3375.4. Samples: 5015748. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:25:46,682][45164] Avg episode reward: [(0, '4299.635'), (1, '2571.787')] [2023-09-19 10:25:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 5287936. Throughput: 0: 3390.8, 1: 3390.7. Samples: 5038688. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:25:51,682][45164] Avg episode reward: [(0, '3978.890'), (1, '2558.455')] [2023-09-19 10:25:55,377][45898] Updated weights for policy 1, policy_version 4960 (0.0014) [2023-09-19 10:25:55,377][45899] Updated weights for policy 0, policy_version 5416 (0.0015) [2023-09-19 10:25:56,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.6, 300 sec: 6970.1). Total num frames: 5320704. Throughput: 0: 3366.3, 1: 3366.5. Samples: 5078850. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:25:56,683][45164] Avg episode reward: [(0, '4069.303'), (1, '2336.977')] [2023-09-19 10:25:56,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005424_2777088.pth... [2023-09-19 10:25:56,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004968_2543616.pth... [2023-09-19 10:25:56,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005224_2674688.pth [2023-09-19 10:25:56,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004768_2441216.pth [2023-09-19 10:26:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 5353472. Throughput: 0: 3398.9, 1: 3398.6. Samples: 5121668. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:26:01,682][45164] Avg episode reward: [(0, '3465.857'), (1, '2175.066')] [2023-09-19 10:26:06,513][45899] Updated weights for policy 0, policy_version 5496 (0.0014) [2023-09-19 10:26:06,513][45898] Updated weights for policy 1, policy_version 5040 (0.0014) [2023-09-19 10:26:06,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 5394432. Throughput: 0: 3452.3, 1: 3453.7. Samples: 5144056. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:26:06,682][45164] Avg episode reward: [(0, '3159.721'), (1, '2334.562')] [2023-09-19 10:26:11,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 5427200. Throughput: 0: 3461.7, 1: 3461.8. Samples: 5187386. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:26:11,682][45164] Avg episode reward: [(0, '3325.508'), (1, '2532.345')] [2023-09-19 10:26:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005528_2830336.pth... [2023-09-19 10:26:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005072_2596864.pth... [2023-09-19 10:26:11,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004864_2490368.pth [2023-09-19 10:26:11,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005320_2723840.pth [2023-09-19 10:26:16,682][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 5459968. Throughput: 0: 3428.7, 1: 3428.7. Samples: 5226062. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:26:16,683][45164] Avg episode reward: [(0, '3507.884'), (1, '2337.732')] [2023-09-19 10:26:18,718][45899] Updated weights for policy 0, policy_version 5576 (0.0011) [2023-09-19 10:26:18,719][45898] Updated weights for policy 1, policy_version 5120 (0.0012) [2023-09-19 10:26:21,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 5492736. Throughput: 0: 3477.6, 1: 3477.5. Samples: 5246086. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:26:21,683][45164] Avg episode reward: [(0, '4071.372'), (1, '2189.272')] [2023-09-19 10:26:26,682][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 5525504. Throughput: 0: 3502.2, 1: 3503.2. Samples: 5287718. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:26:26,683][45164] Avg episode reward: [(0, '4421.465'), (1, '1894.342')] [2023-09-19 10:26:26,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005624_2879488.pth... [2023-09-19 10:26:26,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005168_2646016.pth... [2023-09-19 10:26:26,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005424_2777088.pth [2023-09-19 10:26:26,700][45810] Saving new best policy, reward=4421.465! [2023-09-19 10:26:26,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000004968_2543616.pth [2023-09-19 10:26:30,586][45899] Updated weights for policy 0, policy_version 5656 (0.0011) [2023-09-19 10:26:30,586][45898] Updated weights for policy 1, policy_version 5200 (0.0012) [2023-09-19 10:26:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 5558272. Throughput: 0: 3478.9, 1: 3478.8. Samples: 5328846. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:26:31,682][45164] Avg episode reward: [(0, '4706.176'), (1, '1769.142')] [2023-09-19 10:26:31,684][45810] Saving new best policy, reward=4706.176! [2023-09-19 10:26:36,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 5591040. Throughput: 0: 3417.4, 1: 3417.2. Samples: 5346246. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:26:36,682][45164] Avg episode reward: [(0, '4656.739'), (1, '1920.811')] [2023-09-19 10:26:41,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 5623808. Throughput: 0: 3394.0, 1: 3395.0. Samples: 5384350. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:26:41,682][45164] Avg episode reward: [(0, '4097.684'), (1, '2018.826')] [2023-09-19 10:26:41,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005720_2928640.pth... [2023-09-19 10:26:41,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005264_2695168.pth... [2023-09-19 10:26:41,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005072_2596864.pth [2023-09-19 10:26:41,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005528_2830336.pth [2023-09-19 10:26:43,525][45899] Updated weights for policy 0, policy_version 5736 (0.0014) [2023-09-19 10:26:43,525][45898] Updated weights for policy 1, policy_version 5280 (0.0016) [2023-09-19 10:26:46,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 5656576. Throughput: 0: 3392.5, 1: 3392.8. Samples: 5427006. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:26:46,683][45164] Avg episode reward: [(0, '4145.992'), (1, '2151.049')] [2023-09-19 10:26:51,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 5697536. Throughput: 0: 3399.6, 1: 3399.3. Samples: 5450004. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:26:51,682][45164] Avg episode reward: [(0, '3875.124'), (1, '2129.598')] [2023-09-19 10:26:54,633][45899] Updated weights for policy 0, policy_version 5816 (0.0011) [2023-09-19 10:26:54,633][45898] Updated weights for policy 1, policy_version 5360 (0.0010) [2023-09-19 10:26:56,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 5730304. Throughput: 0: 3394.3, 1: 3394.3. Samples: 5492876. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:26:56,682][45164] Avg episode reward: [(0, '3661.610'), (1, '2128.134')] [2023-09-19 10:26:56,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005824_2981888.pth... [2023-09-19 10:26:56,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005368_2748416.pth... [2023-09-19 10:26:56,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005624_2879488.pth [2023-09-19 10:26:56,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005168_2646016.pth [2023-09-19 10:27:01,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 5771264. Throughput: 0: 3451.7, 1: 3452.0. Samples: 5536726. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:27:01,683][45164] Avg episode reward: [(0, '3369.389'), (1, '2251.457')] [2023-09-19 10:27:06,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6690.1, 300 sec: 6942.4). Total num frames: 5795840. Throughput: 0: 3424.6, 1: 3423.8. Samples: 5554264. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:27:06,682][45164] Avg episode reward: [(0, '3126.397'), (1, '2061.607')] [2023-09-19 10:27:06,782][45899] Updated weights for policy 0, policy_version 5896 (0.0011) [2023-09-19 10:27:06,783][45898] Updated weights for policy 1, policy_version 5440 (0.0015) [2023-09-19 10:27:11,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.6, 300 sec: 6970.1). Total num frames: 5836800. Throughput: 0: 3441.8, 1: 3442.0. Samples: 5597492. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:27:11,683][45164] Avg episode reward: [(0, '3035.897'), (1, '2028.488')] [2023-09-19 10:27:11,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005928_3035136.pth... [2023-09-19 10:27:11,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005472_2801664.pth... [2023-09-19 10:27:11,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005264_2695168.pth [2023-09-19 10:27:11,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005720_2928640.pth [2023-09-19 10:27:16,681][45164] Fps is (10 sec: 8191.9, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 5877760. Throughput: 0: 3472.1, 1: 3472.2. Samples: 5641338. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:27:16,683][45164] Avg episode reward: [(0, '3115.386'), (1, '2059.566')] [2023-09-19 10:27:17,633][45898] Updated weights for policy 1, policy_version 5520 (0.0015) [2023-09-19 10:27:17,633][45899] Updated weights for policy 0, policy_version 5976 (0.0013) [2023-09-19 10:27:21,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 5910528. Throughput: 0: 3536.0, 1: 3536.4. Samples: 5664504. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:27:21,682][45164] Avg episode reward: [(0, '2996.605'), (1, '2133.101')] [2023-09-19 10:27:26,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6997.9). Total num frames: 5951488. Throughput: 0: 3617.0, 1: 3615.7. Samples: 5709822. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:27:26,683][45164] Avg episode reward: [(0, '2768.836'), (1, '2466.188')] [2023-09-19 10:27:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005584_2859008.pth... [2023-09-19 10:27:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006040_3092480.pth... [2023-09-19 10:27:26,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005368_2748416.pth [2023-09-19 10:27:26,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005824_2981888.pth [2023-09-19 10:27:28,787][45898] Updated weights for policy 1, policy_version 5600 (0.0015) [2023-09-19 10:27:28,787][45899] Updated weights for policy 0, policy_version 6056 (0.0013) [2023-09-19 10:27:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.8, 300 sec: 6997.9). Total num frames: 5984256. Throughput: 0: 3616.6, 1: 3616.7. Samples: 5752502. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:27:31,682][45164] Avg episode reward: [(0, '2632.845'), (1, '2613.879')] [2023-09-19 10:27:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.3, 300 sec: 7025.7). Total num frames: 6025216. Throughput: 0: 3616.4, 1: 3615.2. Samples: 5775424. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:27:36,683][45164] Avg episode reward: [(0, '2419.939'), (1, '2879.821')] [2023-09-19 10:27:36,684][45811] Saving new best policy, reward=2879.821! [2023-09-19 10:27:39,510][45898] Updated weights for policy 1, policy_version 5680 (0.0015) [2023-09-19 10:27:39,512][45899] Updated weights for policy 0, policy_version 6136 (0.0015) [2023-09-19 10:27:41,682][45164] Fps is (10 sec: 7372.5, 60 sec: 7236.2, 300 sec: 7025.7). Total num frames: 6057984. Throughput: 0: 3652.6, 1: 3652.7. Samples: 5821618. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:27:41,683][45164] Avg episode reward: [(0, '2463.386'), (1, '2888.804')] [2023-09-19 10:27:41,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005688_2912256.pth... [2023-09-19 10:27:41,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006144_3145728.pth... [2023-09-19 10:27:41,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005472_2801664.pth [2023-09-19 10:27:41,697][45811] Saving new best policy, reward=2888.804! [2023-09-19 10:27:41,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000005928_3035136.pth [2023-09-19 10:27:46,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7236.3, 300 sec: 6997.9). Total num frames: 6090752. Throughput: 0: 3604.7, 1: 3604.8. Samples: 5861154. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:27:46,683][45164] Avg episode reward: [(0, '2769.712'), (1, '2988.841')] [2023-09-19 10:27:46,737][45811] Saving new best policy, reward=2988.841! [2023-09-19 10:27:51,126][45898] Updated weights for policy 1, policy_version 5760 (0.0014) [2023-09-19 10:27:51,126][45899] Updated weights for policy 0, policy_version 6216 (0.0014) [2023-09-19 10:27:51,681][45164] Fps is (10 sec: 7373.1, 60 sec: 7236.3, 300 sec: 7025.7). Total num frames: 6131712. Throughput: 0: 3671.6, 1: 3672.4. Samples: 5884746. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:27:51,682][45164] Avg episode reward: [(0, '3151.986'), (1, '3060.461')] [2023-09-19 10:27:51,684][45811] Saving new best policy, reward=3060.461! [2023-09-19 10:27:56,682][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 7025.7). Total num frames: 6164480. Throughput: 0: 3614.2, 1: 3613.1. Samples: 5922720. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:27:56,682][45164] Avg episode reward: [(0, '3428.935'), (1, '3204.487')] [2023-09-19 10:27:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005792_2965504.pth... [2023-09-19 10:27:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006248_3198976.pth... [2023-09-19 10:27:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005584_2859008.pth [2023-09-19 10:27:56,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006040_3092480.pth [2023-09-19 10:27:56,700][45811] Saving new best policy, reward=3204.487! [2023-09-19 10:28:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7099.7, 300 sec: 7025.7). Total num frames: 6197248. Throughput: 0: 3589.6, 1: 3589.2. Samples: 5964384. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:28:01,682][45164] Avg episode reward: [(0, '3415.287'), (1, '3293.297')] [2023-09-19 10:28:01,684][45811] Saving new best policy, reward=3293.297! [2023-09-19 10:28:03,598][45899] Updated weights for policy 0, policy_version 6296 (0.0012) [2023-09-19 10:28:03,598][45898] Updated weights for policy 1, policy_version 5840 (0.0014) [2023-09-19 10:28:06,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7236.3, 300 sec: 6997.9). Total num frames: 6230016. Throughput: 0: 3557.2, 1: 3557.3. Samples: 5984656. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:28:06,683][45164] Avg episode reward: [(0, '3350.794'), (1, '3083.567')] [2023-09-19 10:28:11,681][45164] Fps is (10 sec: 7373.0, 60 sec: 7236.3, 300 sec: 6997.9). Total num frames: 6270976. Throughput: 0: 3530.6, 1: 3531.9. Samples: 6027636. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:28:11,682][45164] Avg episode reward: [(0, '3598.828'), (1, '2829.130')] [2023-09-19 10:28:11,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006352_3252224.pth... [2023-09-19 10:28:11,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005896_3018752.pth... [2023-09-19 10:28:11,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006144_3145728.pth [2023-09-19 10:28:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005688_2912256.pth [2023-09-19 10:28:15,552][45898] Updated weights for policy 1, policy_version 5920 (0.0015) [2023-09-19 10:28:15,552][45899] Updated weights for policy 0, policy_version 6376 (0.0015) [2023-09-19 10:28:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 6295552. Throughput: 0: 3495.6, 1: 3495.5. Samples: 6067100. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:28:16,682][45164] Avg episode reward: [(0, '3645.754'), (1, '2743.587')] [2023-09-19 10:28:21,681][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 6336512. Throughput: 0: 3487.4, 1: 3488.8. Samples: 6089350. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:28:21,682][45164] Avg episode reward: [(0, '3917.399'), (1, '2849.843')] [2023-09-19 10:28:26,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 6369280. Throughput: 0: 3431.9, 1: 3431.7. Samples: 6130480. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:28:26,682][45164] Avg episode reward: [(0, '3895.625'), (1, '3023.873')] [2023-09-19 10:28:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006448_3301376.pth... [2023-09-19 10:28:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005992_3067904.pth... [2023-09-19 10:28:26,695][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006248_3198976.pth [2023-09-19 10:28:26,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005792_2965504.pth [2023-09-19 10:28:27,485][45898] Updated weights for policy 1, policy_version 6000 (0.0014) [2023-09-19 10:28:27,485][45899] Updated weights for policy 0, policy_version 6456 (0.0012) [2023-09-19 10:28:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 6402048. Throughput: 0: 3420.9, 1: 3420.4. Samples: 6169008. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:28:31,682][45164] Avg episode reward: [(0, '3762.671'), (1, '2994.947')] [2023-09-19 10:28:36,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 6434816. Throughput: 0: 3374.2, 1: 3374.4. Samples: 6188434. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:28:36,682][45164] Avg episode reward: [(0, '3960.535'), (1, '3118.428')] [2023-09-19 10:28:39,026][45898] Updated weights for policy 1, policy_version 6080 (0.0013) [2023-09-19 10:28:39,027][45899] Updated weights for policy 0, policy_version 6536 (0.0013) [2023-09-19 10:28:41,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 6475776. Throughput: 0: 3466.4, 1: 3467.5. Samples: 6234746. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:28:41,682][45164] Avg episode reward: [(0, '3851.079'), (1, '2928.762')] [2023-09-19 10:28:41,686][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006552_3354624.pth... [2023-09-19 10:28:41,687][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006096_3121152.pth... [2023-09-19 10:28:41,693][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005896_3018752.pth [2023-09-19 10:28:41,693][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006352_3252224.pth [2023-09-19 10:28:46,681][45164] Fps is (10 sec: 8192.0, 60 sec: 7099.7, 300 sec: 6997.9). Total num frames: 6516736. Throughput: 0: 3504.8, 1: 3505.2. Samples: 6279834. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:28:46,682][45164] Avg episode reward: [(0, '3917.543'), (1, '2682.745')] [2023-09-19 10:28:49,671][45899] Updated weights for policy 0, policy_version 6616 (0.0013) [2023-09-19 10:28:49,673][45898] Updated weights for policy 1, policy_version 6160 (0.0015) [2023-09-19 10:28:51,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 6549504. Throughput: 0: 3546.9, 1: 3546.8. Samples: 6303874. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:28:51,682][45164] Avg episode reward: [(0, '3789.091'), (1, '2435.076')] [2023-09-19 10:28:56,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 6582272. Throughput: 0: 3554.9, 1: 3554.7. Samples: 6347568. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:28:56,682][45164] Avg episode reward: [(0, '3764.468'), (1, '2480.521')] [2023-09-19 10:28:56,703][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006208_3178496.pth... [2023-09-19 10:28:56,704][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006664_3411968.pth... [2023-09-19 10:28:56,706][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000005992_3067904.pth [2023-09-19 10:28:56,709][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006448_3301376.pth [2023-09-19 10:29:01,174][45898] Updated weights for policy 1, policy_version 6240 (0.0015) [2023-09-19 10:29:01,175][45899] Updated weights for policy 0, policy_version 6696 (0.0015) [2023-09-19 10:29:01,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6997.9). Total num frames: 6623232. Throughput: 0: 3585.8, 1: 3584.5. Samples: 6389764. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:29:01,682][45164] Avg episode reward: [(0, '3392.291'), (1, '2514.234')] [2023-09-19 10:29:06,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6997.9). Total num frames: 6656000. Throughput: 0: 3581.9, 1: 3581.9. Samples: 6411720. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:29:06,682][45164] Avg episode reward: [(0, '3191.423'), (1, '2366.511')] [2023-09-19 10:29:11,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 6688768. Throughput: 0: 3570.8, 1: 3570.8. Samples: 6451854. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:29:11,683][45164] Avg episode reward: [(0, '3030.155'), (1, '2188.160')] [2023-09-19 10:29:11,695][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006760_3461120.pth... [2023-09-19 10:29:11,695][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006304_3227648.pth... [2023-09-19 10:29:11,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006552_3354624.pth [2023-09-19 10:29:11,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006096_3121152.pth [2023-09-19 10:29:13,348][45898] Updated weights for policy 1, policy_version 6320 (0.0011) [2023-09-19 10:29:13,349][45899] Updated weights for policy 0, policy_version 6776 (0.0016) [2023-09-19 10:29:16,682][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 6997.9). Total num frames: 6721536. Throughput: 0: 3583.4, 1: 3583.8. Samples: 6491534. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:29:16,683][45164] Avg episode reward: [(0, '3381.233'), (1, '2144.588')] [2023-09-19 10:29:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 6754304. Throughput: 0: 3593.1, 1: 3593.0. Samples: 6511808. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:29:21,682][45164] Avg episode reward: [(0, '3386.878'), (1, '2294.890')] [2023-09-19 10:29:26,158][45898] Updated weights for policy 1, policy_version 6400 (0.0012) [2023-09-19 10:29:26,158][45899] Updated weights for policy 0, policy_version 6856 (0.0012) [2023-09-19 10:29:26,682][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 6787072. Throughput: 0: 3462.3, 1: 3462.3. Samples: 6546356. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:29:26,683][45164] Avg episode reward: [(0, '3650.732'), (1, '2224.917')] [2023-09-19 10:29:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006856_3510272.pth... [2023-09-19 10:29:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006400_3276800.pth... [2023-09-19 10:29:26,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006664_3411968.pth [2023-09-19 10:29:26,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006208_3178496.pth [2023-09-19 10:29:31,682][45164] Fps is (10 sec: 7372.2, 60 sec: 7099.6, 300 sec: 6997.9). Total num frames: 6828032. Throughput: 0: 3473.5, 1: 3473.5. Samples: 6592456. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:29:31,684][45164] Avg episode reward: [(0, '3727.548'), (1, '2211.809')] [2023-09-19 10:29:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.8, 300 sec: 6997.9). Total num frames: 6860800. Throughput: 0: 3435.7, 1: 3435.8. Samples: 6613092. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:29:36,682][45164] Avg episode reward: [(0, '3915.104'), (1, '2507.415')] [2023-09-19 10:29:37,137][45899] Updated weights for policy 0, policy_version 6936 (0.0015) [2023-09-19 10:29:37,137][45898] Updated weights for policy 1, policy_version 6480 (0.0011) [2023-09-19 10:29:41,681][45164] Fps is (10 sec: 6554.0, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 6893568. Throughput: 0: 3449.0, 1: 3449.4. Samples: 6657996. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:29:41,683][45164] Avg episode reward: [(0, '4009.863'), (1, '2610.333')] [2023-09-19 10:29:41,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006960_3563520.pth... [2023-09-19 10:29:41,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006504_3330048.pth... [2023-09-19 10:29:41,702][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006304_3227648.pth [2023-09-19 10:29:41,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006760_3461120.pth [2023-09-19 10:29:46,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 6934528. Throughput: 0: 3424.0, 1: 3425.3. Samples: 6697984. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:29:46,683][45164] Avg episode reward: [(0, '4265.974'), (1, '2558.952')] [2023-09-19 10:29:48,704][45899] Updated weights for policy 0, policy_version 7016 (0.0012) [2023-09-19 10:29:48,705][45898] Updated weights for policy 1, policy_version 6560 (0.0014) [2023-09-19 10:29:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 6967296. Throughput: 0: 3437.4, 1: 3437.5. Samples: 6721090. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:29:51,682][45164] Avg episode reward: [(0, '4346.801'), (1, '2621.754')] [2023-09-19 10:29:56,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 7000064. Throughput: 0: 3437.5, 1: 3437.6. Samples: 6761234. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:29:56,683][45164] Avg episode reward: [(0, '4557.766'), (1, '2679.788')] [2023-09-19 10:29:56,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007064_3616768.pth... [2023-09-19 10:29:56,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006608_3383296.pth... [2023-09-19 10:29:56,702][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006400_3276800.pth [2023-09-19 10:29:56,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006856_3510272.pth [2023-09-19 10:30:00,732][45898] Updated weights for policy 1, policy_version 6640 (0.0015) [2023-09-19 10:30:00,732][45899] Updated weights for policy 0, policy_version 7096 (0.0015) [2023-09-19 10:30:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 7032832. Throughput: 0: 3452.4, 1: 3452.0. Samples: 6802234. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:30:01,682][45164] Avg episode reward: [(0, '4209.456'), (1, '2765.515')] [2023-09-19 10:30:06,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 7065600. Throughput: 0: 3401.5, 1: 3401.2. Samples: 6817930. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:30:06,682][45164] Avg episode reward: [(0, '3883.678'), (1, '2582.000')] [2023-09-19 10:30:11,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 7098368. Throughput: 0: 3476.9, 1: 3476.9. Samples: 6859278. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:30:11,682][45164] Avg episode reward: [(0, '3625.519'), (1, '2558.234')] [2023-09-19 10:30:11,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007160_3665920.pth... [2023-09-19 10:30:11,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006704_3432448.pth... [2023-09-19 10:30:11,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000006960_3563520.pth [2023-09-19 10:30:11,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006504_3330048.pth [2023-09-19 10:30:13,164][45899] Updated weights for policy 0, policy_version 7176 (0.0015) [2023-09-19 10:30:13,164][45898] Updated weights for policy 1, policy_version 6720 (0.0012) [2023-09-19 10:30:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 7131136. Throughput: 0: 3441.8, 1: 3441.9. Samples: 6902216. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:30:16,683][45164] Avg episode reward: [(0, '3580.654'), (1, '2645.653')] [2023-09-19 10:30:21,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 7172096. Throughput: 0: 3435.6, 1: 3434.4. Samples: 6922242. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:30:21,682][45164] Avg episode reward: [(0, '3964.075'), (1, '2646.071')] [2023-09-19 10:30:24,861][45898] Updated weights for policy 1, policy_version 6800 (0.0013) [2023-09-19 10:30:24,861][45899] Updated weights for policy 0, policy_version 7256 (0.0011) [2023-09-19 10:30:26,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 7204864. Throughput: 0: 3416.9, 1: 3416.7. Samples: 6965506. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:30:26,682][45164] Avg episode reward: [(0, '4267.865'), (1, '2871.914')] [2023-09-19 10:30:26,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007264_3719168.pth... [2023-09-19 10:30:26,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006808_3485696.pth... [2023-09-19 10:30:26,693][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006608_3383296.pth [2023-09-19 10:30:26,693][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007064_3616768.pth [2023-09-19 10:30:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.3, 300 sec: 7025.7). Total num frames: 7245824. Throughput: 0: 3476.0, 1: 3475.8. Samples: 7010814. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:30:31,682][45164] Avg episode reward: [(0, '4442.956'), (1, '2942.457')] [2023-09-19 10:30:35,970][45898] Updated weights for policy 1, policy_version 6880 (0.0015) [2023-09-19 10:30:35,970][45899] Updated weights for policy 0, policy_version 7336 (0.0013) [2023-09-19 10:30:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 7278592. Throughput: 0: 3450.5, 1: 3450.4. Samples: 7031632. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:30:36,682][45164] Avg episode reward: [(0, '4583.012'), (1, '2854.762')] [2023-09-19 10:30:41,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 7025.7). Total num frames: 7319552. Throughput: 0: 3517.5, 1: 3516.3. Samples: 7077756. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:30:41,682][45164] Avg episode reward: [(0, '4746.626'), (1, '2721.347')] [2023-09-19 10:30:41,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007376_3776512.pth... [2023-09-19 10:30:41,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006920_3543040.pth... [2023-09-19 10:30:41,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006704_3432448.pth [2023-09-19 10:30:41,704][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007160_3665920.pth [2023-09-19 10:30:41,704][45810] Saving new best policy, reward=4746.626! [2023-09-19 10:30:46,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 7352320. Throughput: 0: 3483.3, 1: 3483.6. Samples: 7115744. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:30:46,682][45164] Avg episode reward: [(0, '4731.995'), (1, '2652.895')] [2023-09-19 10:30:47,670][45898] Updated weights for policy 1, policy_version 6960 (0.0014) [2023-09-19 10:30:47,671][45899] Updated weights for policy 0, policy_version 7416 (0.0013) [2023-09-19 10:30:51,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6997.9). Total num frames: 7385088. Throughput: 0: 3546.1, 1: 3546.4. Samples: 7137090. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:30:51,682][45164] Avg episode reward: [(0, '4776.263'), (1, '2747.855')] [2023-09-19 10:30:51,683][45810] Saving new best policy, reward=4776.263! [2023-09-19 10:30:56,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 7409664. Throughput: 0: 3481.6, 1: 3481.2. Samples: 7172604. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:30:56,683][45164] Avg episode reward: [(0, '4686.878'), (1, '2798.030')] [2023-09-19 10:30:56,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007464_3821568.pth... [2023-09-19 10:30:56,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007008_3588096.pth... [2023-09-19 10:30:56,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006808_3485696.pth [2023-09-19 10:30:56,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007264_3719168.pth [2023-09-19 10:31:00,367][45898] Updated weights for policy 1, policy_version 7040 (0.0012) [2023-09-19 10:31:00,368][45899] Updated weights for policy 0, policy_version 7496 (0.0015) [2023-09-19 10:31:01,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 7442432. Throughput: 0: 3469.5, 1: 3469.3. Samples: 7214462. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:31:01,682][45164] Avg episode reward: [(0, '4830.372'), (1, '2976.557')] [2023-09-19 10:31:01,683][45810] Saving new best policy, reward=4830.372! [2023-09-19 10:31:06,681][45164] Fps is (10 sec: 6553.8, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 7475200. Throughput: 0: 3435.9, 1: 3436.8. Samples: 7231514. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:06,682][45164] Avg episode reward: [(0, '4711.234'), (1, '3232.713')] [2023-09-19 10:31:11,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 7507968. Throughput: 0: 3351.7, 1: 3351.6. Samples: 7267154. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:11,682][45164] Avg episode reward: [(0, '4670.854'), (1, '3430.314')] [2023-09-19 10:31:11,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007560_3870720.pth... [2023-09-19 10:31:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007104_3637248.pth... [2023-09-19 10:31:11,702][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000006920_3543040.pth [2023-09-19 10:31:11,702][45811] Saving new best policy, reward=3430.314! [2023-09-19 10:31:11,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007376_3776512.pth [2023-09-19 10:31:13,568][45899] Updated weights for policy 0, policy_version 7576 (0.0013) [2023-09-19 10:31:13,569][45898] Updated weights for policy 1, policy_version 7120 (0.0012) [2023-09-19 10:31:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 7540736. Throughput: 0: 3301.1, 1: 3301.3. Samples: 7307918. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:16,682][45164] Avg episode reward: [(0, '4491.464'), (1, '3553.364')] [2023-09-19 10:31:16,682][45811] Saving new best policy, reward=3553.364! [2023-09-19 10:31:21,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 7581696. Throughput: 0: 3335.2, 1: 3334.5. Samples: 7331768. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:21,682][45164] Avg episode reward: [(0, '4544.843'), (1, '3648.952')] [2023-09-19 10:31:21,683][45811] Saving new best policy, reward=3648.952! [2023-09-19 10:31:24,847][45899] Updated weights for policy 0, policy_version 7656 (0.0011) [2023-09-19 10:31:24,848][45898] Updated weights for policy 1, policy_version 7200 (0.0011) [2023-09-19 10:31:26,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 7614464. Throughput: 0: 3304.5, 1: 3305.7. Samples: 7375216. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:26,682][45164] Avg episode reward: [(0, '4484.516'), (1, '3351.888')] [2023-09-19 10:31:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007664_3923968.pth... [2023-09-19 10:31:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007208_3690496.pth... [2023-09-19 10:31:26,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007008_3588096.pth [2023-09-19 10:31:26,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007464_3821568.pth [2023-09-19 10:31:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 6997.9). Total num frames: 7655424. Throughput: 0: 3403.6, 1: 3402.4. Samples: 7422018. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:31,683][45164] Avg episode reward: [(0, '4274.814'), (1, '3158.723')] [2023-09-19 10:31:35,859][45899] Updated weights for policy 0, policy_version 7736 (0.0012) [2023-09-19 10:31:35,859][45898] Updated weights for policy 1, policy_version 7280 (0.0014) [2023-09-19 10:31:36,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.6, 300 sec: 6997.9). Total num frames: 7688192. Throughput: 0: 3400.9, 1: 3401.2. Samples: 7443186. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:36,682][45164] Avg episode reward: [(0, '4304.777'), (1, '3181.465')] [2023-09-19 10:31:41,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 7025.7). Total num frames: 7729152. Throughput: 0: 3493.4, 1: 3493.7. Samples: 7487026. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:31:41,682][45164] Avg episode reward: [(0, '4225.094'), (1, '3295.541')] [2023-09-19 10:31:41,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007776_3981312.pth... [2023-09-19 10:31:41,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007320_3747840.pth... [2023-09-19 10:31:41,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007104_3637248.pth [2023-09-19 10:31:41,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007560_3870720.pth [2023-09-19 10:31:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6997.9). Total num frames: 7761920. Throughput: 0: 3477.3, 1: 3477.3. Samples: 7527420. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:46,683][45164] Avg episode reward: [(0, '4210.255'), (1, '3323.400')] [2023-09-19 10:31:47,493][45899] Updated weights for policy 0, policy_version 7816 (0.0014) [2023-09-19 10:31:47,493][45898] Updated weights for policy 1, policy_version 7360 (0.0014) [2023-09-19 10:31:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.6, 300 sec: 6997.9). Total num frames: 7794688. Throughput: 0: 3521.4, 1: 3521.7. Samples: 7548454. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:51,683][45164] Avg episode reward: [(0, '4314.175'), (1, '3430.675')] [2023-09-19 10:31:56,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 7827456. Throughput: 0: 3537.8, 1: 3537.0. Samples: 7585520. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:31:56,683][45164] Avg episode reward: [(0, '4304.048'), (1, '3346.652')] [2023-09-19 10:31:56,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007872_4030464.pth... [2023-09-19 10:31:56,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007416_3796992.pth... [2023-09-19 10:31:56,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007664_3923968.pth [2023-09-19 10:31:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007208_3690496.pth [2023-09-19 10:32:01,086][45898] Updated weights for policy 1, policy_version 7440 (0.0013) [2023-09-19 10:32:01,087][45899] Updated weights for policy 0, policy_version 7896 (0.0012) [2023-09-19 10:32:01,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 7852032. Throughput: 0: 3454.5, 1: 3454.1. Samples: 7618806. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:32:01,682][45164] Avg episode reward: [(0, '4399.698'), (1, '3356.810')] [2023-09-19 10:32:06,681][45164] Fps is (10 sec: 5734.5, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 7884800. Throughput: 0: 3412.9, 1: 3413.6. Samples: 7638962. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:32:06,682][45164] Avg episode reward: [(0, '4591.203'), (1, '3563.901')] [2023-09-19 10:32:11,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 7917568. Throughput: 0: 3401.8, 1: 3401.6. Samples: 7681368. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:32:11,682][45164] Avg episode reward: [(0, '4544.344'), (1, '3608.968')] [2023-09-19 10:32:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007960_4075520.pth... [2023-09-19 10:32:11,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007504_3842048.pth... [2023-09-19 10:32:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007320_3747840.pth [2023-09-19 10:32:11,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007776_3981312.pth [2023-09-19 10:32:13,867][45898] Updated weights for policy 1, policy_version 7520 (0.0010) [2023-09-19 10:32:13,868][45899] Updated weights for policy 0, policy_version 7976 (0.0012) [2023-09-19 10:32:16,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6690.1, 300 sec: 6886.8). Total num frames: 7942144. Throughput: 0: 3225.6, 1: 3226.3. Samples: 7712354. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:32:16,682][45164] Avg episode reward: [(0, '4574.777'), (1, '3652.918')] [2023-09-19 10:32:16,684][45811] Saving new best policy, reward=3652.918! [2023-09-19 10:32:21,681][45164] Fps is (10 sec: 5734.5, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 7974912. Throughput: 0: 3176.0, 1: 3175.9. Samples: 7729020. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:32:21,682][45164] Avg episode reward: [(0, '4448.116'), (1, '3728.733')] [2023-09-19 10:32:21,683][45811] Saving new best policy, reward=3728.733! [2023-09-19 10:32:26,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 8007680. Throughput: 0: 3156.6, 1: 3156.4. Samples: 7771110. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:32:26,682][45164] Avg episode reward: [(0, '4201.409'), (1, '3581.343')] [2023-09-19 10:32:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008048_4120576.pth... [2023-09-19 10:32:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007592_3887104.pth... [2023-09-19 10:32:26,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007872_4030464.pth [2023-09-19 10:32:26,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007416_3796992.pth [2023-09-19 10:32:26,769][45899] Updated weights for policy 0, policy_version 8056 (0.0013) [2023-09-19 10:32:26,770][45898] Updated weights for policy 1, policy_version 7600 (0.0011) [2023-09-19 10:32:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 8048640. Throughput: 0: 3198.4, 1: 3197.4. Samples: 7815232. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:32:31,682][45164] Avg episode reward: [(0, '4070.275'), (1, '3528.817')] [2023-09-19 10:32:36,682][45164] Fps is (10 sec: 7372.8, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 8081408. Throughput: 0: 3213.7, 1: 3213.8. Samples: 7837692. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:32:36,682][45164] Avg episode reward: [(0, '4319.055'), (1, '3470.292')] [2023-09-19 10:32:38,117][45899] Updated weights for policy 0, policy_version 8136 (0.0014) [2023-09-19 10:32:38,118][45898] Updated weights for policy 1, policy_version 7680 (0.0013) [2023-09-19 10:32:41,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6886.8). Total num frames: 8122368. Throughput: 0: 3276.2, 1: 3276.0. Samples: 7880372. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:32:41,683][45164] Avg episode reward: [(0, '4566.377'), (1, '3648.534')] [2023-09-19 10:32:41,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007704_3944448.pth... [2023-09-19 10:32:41,694][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008160_4177920.pth... [2023-09-19 10:32:41,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007504_3842048.pth [2023-09-19 10:32:41,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000007960_4075520.pth [2023-09-19 10:32:46,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6831.3). Total num frames: 8146944. Throughput: 0: 3277.4, 1: 3277.7. Samples: 7913784. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:32:46,682][45164] Avg episode reward: [(0, '4610.564'), (1, '3500.855')] [2023-09-19 10:32:50,281][45899] Updated weights for policy 0, policy_version 8216 (0.0014) [2023-09-19 10:32:50,281][45898] Updated weights for policy 1, policy_version 7760 (0.0014) [2023-09-19 10:32:51,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 8187904. Throughput: 0: 3320.6, 1: 3320.7. Samples: 7937820. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:32:51,682][45164] Avg episode reward: [(0, '4766.034'), (1, '3662.622')] [2023-09-19 10:32:56,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 8220672. Throughput: 0: 3362.1, 1: 3361.7. Samples: 7983936. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:32:56,682][45164] Avg episode reward: [(0, '4942.506'), (1, '3426.518')] [2023-09-19 10:32:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008256_4227072.pth... [2023-09-19 10:32:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007800_3993600.pth... [2023-09-19 10:32:56,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008048_4120576.pth [2023-09-19 10:32:56,698][45810] Saving new best policy, reward=4942.506! [2023-09-19 10:32:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007592_3887104.pth [2023-09-19 10:33:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 8253440. Throughput: 0: 3472.7, 1: 3472.9. Samples: 8024906. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:33:01,682][45164] Avg episode reward: [(0, '4914.631'), (1, '3484.643')] [2023-09-19 10:33:01,854][45899] Updated weights for policy 0, policy_version 8296 (0.0014) [2023-09-19 10:33:01,854][45898] Updated weights for policy 1, policy_version 7840 (0.0016) [2023-09-19 10:33:06,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 8294400. Throughput: 0: 3507.2, 1: 3506.0. Samples: 8044616. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:33:06,682][45164] Avg episode reward: [(0, '4850.954'), (1, '3317.123')] [2023-09-19 10:33:11,682][45164] Fps is (10 sec: 8191.8, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 8335360. Throughput: 0: 3572.7, 1: 3572.8. Samples: 8092656. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:33:11,683][45164] Avg episode reward: [(0, '4524.288'), (1, '3312.081')] [2023-09-19 10:33:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008368_4284416.pth... [2023-09-19 10:33:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007912_4050944.pth... [2023-09-19 10:33:11,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008160_4177920.pth [2023-09-19 10:33:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007704_3944448.pth [2023-09-19 10:33:12,930][45899] Updated weights for policy 0, policy_version 8376 (0.0016) [2023-09-19 10:33:12,930][45898] Updated weights for policy 1, policy_version 7920 (0.0016) [2023-09-19 10:33:16,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6886.8). Total num frames: 8368128. Throughput: 0: 3549.2, 1: 3549.2. Samples: 8134658. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:33:16,682][45164] Avg episode reward: [(0, '4460.794'), (1, '3323.627')] [2023-09-19 10:33:21,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.2, 300 sec: 6914.6). Total num frames: 8409088. Throughput: 0: 3568.9, 1: 3569.0. Samples: 8158898. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:33:21,683][45164] Avg episode reward: [(0, '4434.022'), (1, '3041.799')] [2023-09-19 10:33:24,070][45899] Updated weights for policy 0, policy_version 8456 (0.0011) [2023-09-19 10:33:24,070][45898] Updated weights for policy 1, policy_version 8000 (0.0016) [2023-09-19 10:33:26,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 6914.6). Total num frames: 8441856. Throughput: 0: 3554.4, 1: 3554.2. Samples: 8200258. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:33:26,682][45164] Avg episode reward: [(0, '4669.100'), (1, '2791.116')] [2023-09-19 10:33:26,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008016_4104192.pth... [2023-09-19 10:33:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008472_4337664.pth... [2023-09-19 10:33:26,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007800_3993600.pth [2023-09-19 10:33:26,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008256_4227072.pth [2023-09-19 10:33:31,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.3, 300 sec: 6942.4). Total num frames: 8482816. Throughput: 0: 3710.2, 1: 3710.1. Samples: 8247694. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:33:31,682][45164] Avg episode reward: [(0, '4460.698'), (1, '2397.094')] [2023-09-19 10:33:34,656][45899] Updated weights for policy 0, policy_version 8536 (0.0012) [2023-09-19 10:33:34,656][45898] Updated weights for policy 1, policy_version 8080 (0.0013) [2023-09-19 10:33:36,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 6914.6). Total num frames: 8515584. Throughput: 0: 3696.4, 1: 3696.4. Samples: 8270498. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:33:36,683][45164] Avg episode reward: [(0, '3710.773'), (1, '2108.783')] [2023-09-19 10:33:41,681][45164] Fps is (10 sec: 7372.6, 60 sec: 7236.3, 300 sec: 6914.6). Total num frames: 8556544. Throughput: 0: 3678.7, 1: 3679.4. Samples: 8315052. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:33:41,683][45164] Avg episode reward: [(0, '3296.558'), (1, '2039.146')] [2023-09-19 10:33:41,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008584_4395008.pth... [2023-09-19 10:33:41,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008128_4161536.pth... [2023-09-19 10:33:41,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008368_4284416.pth [2023-09-19 10:33:41,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000007912_4050944.pth [2023-09-19 10:33:45,941][45898] Updated weights for policy 1, policy_version 8160 (0.0013) [2023-09-19 10:33:45,941][45899] Updated weights for policy 0, policy_version 8616 (0.0015) [2023-09-19 10:33:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7372.8, 300 sec: 6914.6). Total num frames: 8589312. Throughput: 0: 3693.1, 1: 3693.4. Samples: 8357302. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:33:46,682][45164] Avg episode reward: [(0, '3446.255'), (1, '2228.780')] [2023-09-19 10:33:51,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7236.3, 300 sec: 6914.6). Total num frames: 8622080. Throughput: 0: 3724.8, 1: 3726.1. Samples: 8379906. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:33:51,682][45164] Avg episode reward: [(0, '3959.905'), (1, '2143.928')] [2023-09-19 10:33:56,682][45164] Fps is (10 sec: 6553.5, 60 sec: 7236.2, 300 sec: 6886.8). Total num frames: 8654848. Throughput: 0: 3595.9, 1: 3595.9. Samples: 8416286. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:33:56,683][45164] Avg episode reward: [(0, '4216.887'), (1, '2079.015')] [2023-09-19 10:33:56,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008680_4444160.pth... [2023-09-19 10:33:56,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008224_4210688.pth... [2023-09-19 10:33:56,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008016_4104192.pth [2023-09-19 10:33:56,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008472_4337664.pth [2023-09-19 10:33:57,923][45899] Updated weights for policy 0, policy_version 8696 (0.0013) [2023-09-19 10:33:57,924][45898] Updated weights for policy 1, policy_version 8240 (0.0013) [2023-09-19 10:34:01,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7372.8, 300 sec: 6914.6). Total num frames: 8695808. Throughput: 0: 3641.6, 1: 3641.6. Samples: 8462400. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:34:01,682][45164] Avg episode reward: [(0, '4350.168'), (1, '2218.363')] [2023-09-19 10:34:06,681][45164] Fps is (10 sec: 7373.0, 60 sec: 7236.3, 300 sec: 6914.6). Total num frames: 8728576. Throughput: 0: 3588.6, 1: 3588.1. Samples: 8481848. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:34:06,682][45164] Avg episode reward: [(0, '4222.421'), (1, '2284.420')] [2023-09-19 10:34:09,790][45898] Updated weights for policy 1, policy_version 8320 (0.0012) [2023-09-19 10:34:09,791][45899] Updated weights for policy 0, policy_version 8776 (0.0014) [2023-09-19 10:34:11,682][45164] Fps is (10 sec: 6553.4, 60 sec: 7099.7, 300 sec: 6914.6). Total num frames: 8761344. Throughput: 0: 3579.0, 1: 3580.3. Samples: 8522428. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:34:11,683][45164] Avg episode reward: [(0, '4143.659'), (1, '2411.642')] [2023-09-19 10:34:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008784_4497408.pth... [2023-09-19 10:34:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008328_4263936.pth... [2023-09-19 10:34:11,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008584_4395008.pth [2023-09-19 10:34:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008128_4161536.pth [2023-09-19 10:34:16,681][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 6914.6). Total num frames: 8794112. Throughput: 0: 3520.5, 1: 3520.7. Samples: 8564552. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:34:16,683][45164] Avg episode reward: [(0, '4595.055'), (1, '2441.772')] [2023-09-19 10:34:21,473][45899] Updated weights for policy 0, policy_version 8856 (0.0011) [2023-09-19 10:34:21,474][45898] Updated weights for policy 1, policy_version 8400 (0.0017) [2023-09-19 10:34:21,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 8835072. Throughput: 0: 3518.1, 1: 3518.2. Samples: 8587134. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:34:21,682][45164] Avg episode reward: [(0, '4798.156'), (1, '2187.171')] [2023-09-19 10:34:26,682][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6914.6). Total num frames: 8867840. Throughput: 0: 3461.2, 1: 3461.1. Samples: 8626556. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:34:26,683][45164] Avg episode reward: [(0, '5126.605'), (1, '2033.472')] [2023-09-19 10:34:26,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008432_4317184.pth... [2023-09-19 10:34:26,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008888_4550656.pth... [2023-09-19 10:34:26,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008224_4210688.pth [2023-09-19 10:34:26,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008680_4444160.pth [2023-09-19 10:34:26,702][45810] Saving new best policy, reward=5126.605! [2023-09-19 10:34:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 8900608. Throughput: 0: 3455.4, 1: 3455.1. Samples: 8668276. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:34:31,682][45164] Avg episode reward: [(0, '5341.382'), (1, '2100.424')] [2023-09-19 10:34:31,684][45810] Saving new best policy, reward=5341.382! [2023-09-19 10:34:33,652][45898] Updated weights for policy 1, policy_version 8480 (0.0010) [2023-09-19 10:34:33,653][45899] Updated weights for policy 0, policy_version 8936 (0.0016) [2023-09-19 10:34:36,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 8933376. Throughput: 0: 3417.4, 1: 3417.3. Samples: 8687468. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:34:36,682][45164] Avg episode reward: [(0, '5044.752'), (1, '2147.586')] [2023-09-19 10:34:41,682][45164] Fps is (10 sec: 6553.4, 60 sec: 6826.6, 300 sec: 6886.8). Total num frames: 8966144. Throughput: 0: 3423.5, 1: 3422.4. Samples: 8724352. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:34:41,683][45164] Avg episode reward: [(0, '4898.544'), (1, '2177.191')] [2023-09-19 10:34:41,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008528_4366336.pth... [2023-09-19 10:34:41,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008984_4599808.pth... [2023-09-19 10:34:41,702][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008328_4263936.pth [2023-09-19 10:34:41,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008784_4497408.pth [2023-09-19 10:34:46,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 8990720. Throughput: 0: 3289.8, 1: 3290.9. Samples: 8758534. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:34:46,683][45164] Avg episode reward: [(0, '4738.180'), (1, '2293.790')] [2023-09-19 10:34:47,202][45899] Updated weights for policy 0, policy_version 9016 (0.0013) [2023-09-19 10:34:47,202][45898] Updated weights for policy 1, policy_version 8560 (0.0016) [2023-09-19 10:34:51,681][45164] Fps is (10 sec: 6553.8, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 9031680. Throughput: 0: 3331.6, 1: 3331.9. Samples: 8781706. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:34:51,683][45164] Avg episode reward: [(0, '4656.102'), (1, '2370.064')] [2023-09-19 10:34:56,681][45164] Fps is (10 sec: 7373.0, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 9064448. Throughput: 0: 3383.8, 1: 3383.3. Samples: 8826944. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:34:56,682][45164] Avg episode reward: [(0, '4750.063'), (1, '2526.681')] [2023-09-19 10:34:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009080_4648960.pth... [2023-09-19 10:34:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008624_4415488.pth... [2023-09-19 10:34:56,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008888_4550656.pth [2023-09-19 10:34:56,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008432_4317184.pth [2023-09-19 10:34:58,593][45898] Updated weights for policy 1, policy_version 8640 (0.0014) [2023-09-19 10:34:58,593][45899] Updated weights for policy 0, policy_version 9096 (0.0014) [2023-09-19 10:35:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6886.8). Total num frames: 9097216. Throughput: 0: 3338.2, 1: 3338.0. Samples: 8864980. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:35:01,683][45164] Avg episode reward: [(0, '4935.627'), (1, '2660.947')] [2023-09-19 10:35:06,681][45164] Fps is (10 sec: 7372.6, 60 sec: 6826.6, 300 sec: 6914.6). Total num frames: 9138176. Throughput: 0: 3341.6, 1: 3341.3. Samples: 8887864. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:35:06,682][45164] Avg episode reward: [(0, '4716.968'), (1, '2814.567')] [2023-09-19 10:35:10,387][45898] Updated weights for policy 1, policy_version 8720 (0.0011) [2023-09-19 10:35:10,388][45899] Updated weights for policy 0, policy_version 9176 (0.0012) [2023-09-19 10:35:11,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 9170944. Throughput: 0: 3360.7, 1: 3359.7. Samples: 8928976. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:35:11,683][45164] Avg episode reward: [(0, '4783.726'), (1, '2932.118')] [2023-09-19 10:35:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009184_4702208.pth... [2023-09-19 10:35:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008728_4468736.pth... [2023-09-19 10:35:11,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000008984_4599808.pth [2023-09-19 10:35:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008528_4366336.pth [2023-09-19 10:35:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 9203712. Throughput: 0: 3331.1, 1: 3331.3. Samples: 8968086. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:35:16,683][45164] Avg episode reward: [(0, '4843.597'), (1, '2844.457')] [2023-09-19 10:35:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6690.1, 300 sec: 6886.8). Total num frames: 9236480. Throughput: 0: 3362.8, 1: 3362.8. Samples: 8990120. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:35:21,683][45164] Avg episode reward: [(0, '5023.013'), (1, '2819.580')] [2023-09-19 10:35:21,932][45898] Updated weights for policy 1, policy_version 8800 (0.0011) [2023-09-19 10:35:21,932][45899] Updated weights for policy 0, policy_version 9256 (0.0014) [2023-09-19 10:35:26,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 9269248. Throughput: 0: 3431.3, 1: 3432.3. Samples: 9033216. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:35:26,683][45164] Avg episode reward: [(0, '4851.745'), (1, '2994.098')] [2023-09-19 10:35:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009280_4751360.pth... [2023-09-19 10:35:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008824_4517888.pth... [2023-09-19 10:35:26,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008624_4415488.pth [2023-09-19 10:35:26,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009080_4648960.pth [2023-09-19 10:35:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 9310208. Throughput: 0: 3516.0, 1: 3515.9. Samples: 9074968. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:35:31,682][45164] Avg episode reward: [(0, '4571.520'), (1, '3260.056')] [2023-09-19 10:35:33,739][45898] Updated weights for policy 1, policy_version 8880 (0.0013) [2023-09-19 10:35:33,739][45899] Updated weights for policy 0, policy_version 9336 (0.0017) [2023-09-19 10:35:36,681][45164] Fps is (10 sec: 7373.0, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 9342976. Throughput: 0: 3497.3, 1: 3497.4. Samples: 9096468. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:35:36,682][45164] Avg episode reward: [(0, '3058.952'), (1, '3455.869')] [2023-09-19 10:35:41,682][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6886.8). Total num frames: 9383936. Throughput: 0: 3507.0, 1: 3507.2. Samples: 9142584. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:35:41,683][45164] Avg episode reward: [(0, '3095.524'), (1, '3308.335')] [2023-09-19 10:35:41,696][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009392_4808704.pth... [2023-09-19 10:35:41,696][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008936_4575232.pth... [2023-09-19 10:35:41,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008728_4468736.pth [2023-09-19 10:35:41,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009184_4702208.pth [2023-09-19 10:35:44,671][45899] Updated weights for policy 0, policy_version 9416 (0.0012) [2023-09-19 10:35:44,671][45898] Updated weights for policy 1, policy_version 8960 (0.0013) [2023-09-19 10:35:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.8, 300 sec: 6886.8). Total num frames: 9416704. Throughput: 0: 3565.2, 1: 3565.2. Samples: 9185848. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:35:46,682][45164] Avg episode reward: [(0, '3287.864'), (1, '3397.982')] [2023-09-19 10:35:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 9457664. Throughput: 0: 3555.2, 1: 3554.3. Samples: 9207794. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:35:51,683][45164] Avg episode reward: [(0, '3764.475'), (1, '3294.614')] [2023-09-19 10:35:56,169][45898] Updated weights for policy 1, policy_version 9040 (0.0009) [2023-09-19 10:35:56,171][45899] Updated weights for policy 0, policy_version 9496 (0.0013) [2023-09-19 10:35:56,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 9490432. Throughput: 0: 3560.0, 1: 3560.4. Samples: 9249392. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:35:56,682][45164] Avg episode reward: [(0, '4482.686'), (1, '3128.718')] [2023-09-19 10:35:56,687][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009496_4861952.pth... [2023-09-19 10:35:56,688][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009040_4628480.pth... [2023-09-19 10:35:56,691][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009280_4751360.pth [2023-09-19 10:35:56,695][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008824_4517888.pth [2023-09-19 10:36:01,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 9523200. Throughput: 0: 3588.5, 1: 3588.4. Samples: 9291042. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:36:01,682][45164] Avg episode reward: [(0, '5072.014'), (1, '3056.962')] [2023-09-19 10:36:06,681][45164] Fps is (10 sec: 7372.6, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 9564160. Throughput: 0: 3601.8, 1: 3601.2. Samples: 9314256. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:36:06,682][45164] Avg episode reward: [(0, '5340.934'), (1, '3132.790')] [2023-09-19 10:36:07,633][45899] Updated weights for policy 0, policy_version 9576 (0.0015) [2023-09-19 10:36:07,633][45898] Updated weights for policy 1, policy_version 9120 (0.0015) [2023-09-19 10:36:11,682][45164] Fps is (10 sec: 7372.6, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 9596928. Throughput: 0: 3616.4, 1: 3616.5. Samples: 9358700. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:36:11,683][45164] Avg episode reward: [(0, '5452.588'), (1, '3244.621')] [2023-09-19 10:36:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009600_4915200.pth... [2023-09-19 10:36:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009144_4681728.pth... [2023-09-19 10:36:11,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000008936_4575232.pth [2023-09-19 10:36:11,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009392_4808704.pth [2023-09-19 10:36:11,700][45810] Saving new best policy, reward=5452.588! [2023-09-19 10:36:16,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 9629696. Throughput: 0: 3593.7, 1: 3593.6. Samples: 9398396. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:36:16,682][45164] Avg episode reward: [(0, '5258.168'), (1, '3308.573')] [2023-09-19 10:36:20,158][45898] Updated weights for policy 1, policy_version 9200 (0.0014) [2023-09-19 10:36:20,158][45899] Updated weights for policy 0, policy_version 9656 (0.0014) [2023-09-19 10:36:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 9662464. Throughput: 0: 3524.0, 1: 3523.9. Samples: 9413622. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:36:21,683][45164] Avg episode reward: [(0, '5081.302'), (1, '3305.028')] [2023-09-19 10:36:26,682][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 6914.6). Total num frames: 9695232. Throughput: 0: 3481.3, 1: 3481.6. Samples: 9455914. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:36:26,683][45164] Avg episode reward: [(0, '4867.897'), (1, '3443.833')] [2023-09-19 10:36:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009240_4730880.pth... [2023-09-19 10:36:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009696_4964352.pth... [2023-09-19 10:36:26,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009040_4628480.pth [2023-09-19 10:36:26,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009496_4861952.pth [2023-09-19 10:36:31,264][45899] Updated weights for policy 0, policy_version 9736 (0.0014) [2023-09-19 10:36:31,264][45898] Updated weights for policy 1, policy_version 9280 (0.0014) [2023-09-19 10:36:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 9736192. Throughput: 0: 3521.0, 1: 3519.9. Samples: 9502692. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:36:31,682][45164] Avg episode reward: [(0, '5118.954'), (1, '3683.277')] [2023-09-19 10:36:36,681][45164] Fps is (10 sec: 7373.0, 60 sec: 7099.7, 300 sec: 6914.6). Total num frames: 9768960. Throughput: 0: 3467.5, 1: 3468.6. Samples: 9519916. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:36:36,682][45164] Avg episode reward: [(0, '5158.754'), (1, '3617.845')] [2023-09-19 10:36:41,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 9801728. Throughput: 0: 3497.9, 1: 3498.4. Samples: 9564224. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:36:41,682][45164] Avg episode reward: [(0, '5112.694'), (1, '3759.050')] [2023-09-19 10:36:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009344_4784128.pth... [2023-09-19 10:36:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009800_5017600.pth... [2023-09-19 10:36:41,695][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009144_4681728.pth [2023-09-19 10:36:41,696][45811] Saving new best policy, reward=3759.050! [2023-09-19 10:36:41,696][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009600_4915200.pth [2023-09-19 10:36:43,531][45899] Updated weights for policy 0, policy_version 9816 (0.0011) [2023-09-19 10:36:43,531][45898] Updated weights for policy 1, policy_version 9360 (0.0016) [2023-09-19 10:36:46,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 9834496. Throughput: 0: 3444.8, 1: 3443.7. Samples: 9601024. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:36:46,682][45164] Avg episode reward: [(0, '5162.424'), (1, '3727.675')] [2023-09-19 10:36:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 9867264. Throughput: 0: 3426.3, 1: 3426.9. Samples: 9622648. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:36:51,682][45164] Avg episode reward: [(0, '5162.492'), (1, '3676.891')] [2023-09-19 10:36:55,799][45899] Updated weights for policy 0, policy_version 9896 (0.0013) [2023-09-19 10:36:55,800][45898] Updated weights for policy 1, policy_version 9440 (0.0015) [2023-09-19 10:36:56,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.6, 300 sec: 6942.4). Total num frames: 9900032. Throughput: 0: 3364.9, 1: 3364.8. Samples: 9661540. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:36:56,683][45164] Avg episode reward: [(0, '5074.296'), (1, '3448.581')] [2023-09-19 10:36:56,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009896_5066752.pth... [2023-09-19 10:36:56,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009440_4833280.pth... [2023-09-19 10:36:56,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009696_4964352.pth [2023-09-19 10:36:56,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009240_4730880.pth [2023-09-19 10:37:01,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 9940992. Throughput: 0: 3441.6, 1: 3441.9. Samples: 9708156. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:37:01,683][45164] Avg episode reward: [(0, '5183.544'), (1, '3211.653')] [2023-09-19 10:37:06,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 9973760. Throughput: 0: 3517.7, 1: 3517.7. Samples: 9730216. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:37:06,683][45164] Avg episode reward: [(0, '5151.022'), (1, '3081.168')] [2023-09-19 10:37:07,784][45898] Updated weights for policy 1, policy_version 9520 (0.0010) [2023-09-19 10:37:07,785][45899] Updated weights for policy 0, policy_version 9976 (0.0014) [2023-09-19 10:37:11,682][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6997.9). Total num frames: 10006528. Throughput: 0: 3443.4, 1: 3443.3. Samples: 9765816. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:37:11,683][45164] Avg episode reward: [(0, '5245.711'), (1, '3050.875')] [2023-09-19 10:37:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009544_4886528.pth... [2023-09-19 10:37:11,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010000_5120000.pth... [2023-09-19 10:37:11,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009800_5017600.pth [2023-09-19 10:37:11,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009344_4784128.pth [2023-09-19 10:37:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6997.9). Total num frames: 10039296. Throughput: 0: 3368.2, 1: 3368.1. Samples: 9805826. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:37:16,683][45164] Avg episode reward: [(0, '5455.017'), (1, '3284.047')] [2023-09-19 10:37:16,684][45810] Saving new best policy, reward=5455.017! [2023-09-19 10:37:19,471][45899] Updated weights for policy 0, policy_version 10056 (0.0015) [2023-09-19 10:37:19,472][45898] Updated weights for policy 1, policy_version 9600 (0.0015) [2023-09-19 10:37:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6997.9). Total num frames: 10072064. Throughput: 0: 3426.4, 1: 3426.4. Samples: 9828294. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:37:21,682][45164] Avg episode reward: [(0, '5497.064'), (1, '3225.451')] [2023-09-19 10:37:21,684][45810] Saving new best policy, reward=5497.064! [2023-09-19 10:37:26,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 10104832. Throughput: 0: 3348.1, 1: 3347.9. Samples: 9865542. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:37:26,683][45164] Avg episode reward: [(0, '5561.339'), (1, '3261.117')] [2023-09-19 10:37:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009640_4935680.pth... [2023-09-19 10:37:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010096_5169152.pth... [2023-09-19 10:37:26,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009440_4833280.pth [2023-09-19 10:37:26,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000009896_5066752.pth [2023-09-19 10:37:26,701][45810] Saving new best policy, reward=5561.339! [2023-09-19 10:37:31,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6690.1, 300 sec: 6970.1). Total num frames: 10137600. Throughput: 0: 3381.8, 1: 3382.7. Samples: 9905428. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:37:31,682][45164] Avg episode reward: [(0, '5541.661'), (1, '3168.130')] [2023-09-19 10:37:32,098][45899] Updated weights for policy 0, policy_version 10136 (0.0010) [2023-09-19 10:37:32,098][45898] Updated weights for policy 1, policy_version 9680 (0.0011) [2023-09-19 10:37:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 10178560. Throughput: 0: 3400.4, 1: 3399.8. Samples: 9928658. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:37:36,682][45164] Avg episode reward: [(0, '5526.103'), (1, '3370.990')] [2023-09-19 10:37:41,682][45164] Fps is (10 sec: 7372.5, 60 sec: 6826.6, 300 sec: 6997.9). Total num frames: 10211328. Throughput: 0: 3458.9, 1: 3459.0. Samples: 9972846. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:37:41,683][45164] Avg episode reward: [(0, '5469.077'), (1, '3490.475')] [2023-09-19 10:37:41,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009744_4988928.pth... [2023-09-19 10:37:41,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010200_5222400.pth... [2023-09-19 10:37:41,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009544_4886528.pth [2023-09-19 10:37:41,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010000_5120000.pth [2023-09-19 10:37:43,238][45899] Updated weights for policy 0, policy_version 10216 (0.0012) [2023-09-19 10:37:43,238][45898] Updated weights for policy 1, policy_version 9760 (0.0014) [2023-09-19 10:37:46,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 10244096. Throughput: 0: 3353.7, 1: 3352.9. Samples: 10009952. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:37:46,682][45164] Avg episode reward: [(0, '5662.775'), (1, '3402.955')] [2023-09-19 10:37:46,684][45810] Saving new best policy, reward=5662.775! [2023-09-19 10:37:51,681][45164] Fps is (10 sec: 5734.6, 60 sec: 6690.1, 300 sec: 6942.4). Total num frames: 10268672. Throughput: 0: 3299.0, 1: 3298.1. Samples: 10027088. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:37:51,682][45164] Avg episode reward: [(0, '5448.957'), (1, '3482.978')] [2023-09-19 10:37:56,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6690.2, 300 sec: 6942.4). Total num frames: 10301440. Throughput: 0: 3276.3, 1: 3276.1. Samples: 10060674. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:37:56,682][45164] Avg episode reward: [(0, '5144.916'), (1, '3568.689')] [2023-09-19 10:37:56,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009832_5033984.pth... [2023-09-19 10:37:56,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010288_5267456.pth... [2023-09-19 10:37:56,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010096_5169152.pth [2023-09-19 10:37:56,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009640_4935680.pth [2023-09-19 10:37:57,376][45899] Updated weights for policy 0, policy_version 10296 (0.0016) [2023-09-19 10:37:57,377][45898] Updated weights for policy 1, policy_version 9840 (0.0014) [2023-09-19 10:38:01,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6914.6). Total num frames: 10334208. Throughput: 0: 3302.2, 1: 3303.5. Samples: 10103080. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:38:01,682][45164] Avg episode reward: [(0, '5055.092'), (1, '3675.966')] [2023-09-19 10:38:06,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6690.1, 300 sec: 6914.6). Total num frames: 10375168. Throughput: 0: 3300.9, 1: 3299.6. Samples: 10125316. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:38:06,683][45164] Avg episode reward: [(0, '4835.377'), (1, '3803.681')] [2023-09-19 10:38:06,684][45811] Saving new best policy, reward=3803.681! [2023-09-19 10:38:08,520][45898] Updated weights for policy 1, policy_version 9920 (0.0014) [2023-09-19 10:38:08,521][45899] Updated weights for policy 0, policy_version 10376 (0.0011) [2023-09-19 10:38:11,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6690.1, 300 sec: 6914.6). Total num frames: 10407936. Throughput: 0: 3390.1, 1: 3390.2. Samples: 10170656. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:38:11,682][45164] Avg episode reward: [(0, '4311.681'), (1, '3774.506')] [2023-09-19 10:38:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009936_5087232.pth... [2023-09-19 10:38:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010392_5320704.pth... [2023-09-19 10:38:11,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010200_5222400.pth [2023-09-19 10:38:11,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009744_4988928.pth [2023-09-19 10:38:16,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 10448896. Throughput: 0: 3439.1, 1: 3439.5. Samples: 10214964. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:38:16,682][45164] Avg episode reward: [(0, '4014.792'), (1, '3443.743')] [2023-09-19 10:38:19,415][45899] Updated weights for policy 0, policy_version 10456 (0.0012) [2023-09-19 10:38:19,416][45898] Updated weights for policy 1, policy_version 10000 (0.0012) [2023-09-19 10:38:21,681][45164] Fps is (10 sec: 8192.0, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 10489856. Throughput: 0: 3439.5, 1: 3440.2. Samples: 10238244. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:38:21,683][45164] Avg episode reward: [(0, '3869.063'), (1, '3353.230')] [2023-09-19 10:38:26,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 10522624. Throughput: 0: 3420.4, 1: 3419.3. Samples: 10280628. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:38:26,683][45164] Avg episode reward: [(0, '3766.379'), (1, '3386.730')] [2023-09-19 10:38:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010504_5378048.pth... [2023-09-19 10:38:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010048_5144576.pth... [2023-09-19 10:38:26,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010288_5267456.pth [2023-09-19 10:38:26,704][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009832_5033984.pth [2023-09-19 10:38:31,506][45898] Updated weights for policy 1, policy_version 10080 (0.0011) [2023-09-19 10:38:31,507][45899] Updated weights for policy 0, policy_version 10536 (0.0014) [2023-09-19 10:38:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 10555392. Throughput: 0: 3452.4, 1: 3453.3. Samples: 10320712. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:38:31,682][45164] Avg episode reward: [(0, '3975.517'), (1, '3321.568')] [2023-09-19 10:38:36,681][45164] Fps is (10 sec: 5734.6, 60 sec: 6690.2, 300 sec: 6859.1). Total num frames: 10579968. Throughput: 0: 3458.2, 1: 3458.6. Samples: 10338338. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:38:36,682][45164] Avg episode reward: [(0, '4436.875'), (1, '3292.402')] [2023-09-19 10:38:41,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 10620928. Throughput: 0: 3565.9, 1: 3566.1. Samples: 10381612. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:38:41,682][45164] Avg episode reward: [(0, '4357.635'), (1, '3252.418')] [2023-09-19 10:38:41,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010600_5427200.pth... [2023-09-19 10:38:41,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010144_5193728.pth... [2023-09-19 10:38:41,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010392_5320704.pth [2023-09-19 10:38:41,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000009936_5087232.pth [2023-09-19 10:38:43,028][45898] Updated weights for policy 1, policy_version 10160 (0.0013) [2023-09-19 10:38:43,028][45899] Updated weights for policy 0, policy_version 10616 (0.0015) [2023-09-19 10:38:46,681][45164] Fps is (10 sec: 7372.6, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 10653696. Throughput: 0: 3585.0, 1: 3584.7. Samples: 10425716. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:38:46,682][45164] Avg episode reward: [(0, '4425.271'), (1, '3245.521')] [2023-09-19 10:38:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6886.8). Total num frames: 10686464. Throughput: 0: 3509.4, 1: 3510.6. Samples: 10441216. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:38:51,682][45164] Avg episode reward: [(0, '4507.979'), (1, '3336.155')] [2023-09-19 10:38:55,934][45899] Updated weights for policy 0, policy_version 10696 (0.0019) [2023-09-19 10:38:55,934][45898] Updated weights for policy 1, policy_version 10240 (0.0020) [2023-09-19 10:38:56,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6859.1). Total num frames: 10719232. Throughput: 0: 3431.8, 1: 3431.9. Samples: 10479522. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:38:56,683][45164] Avg episode reward: [(0, '4439.888'), (1, '3242.988')] [2023-09-19 10:38:56,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010240_5242880.pth... [2023-09-19 10:38:56,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010696_5476352.pth... [2023-09-19 10:38:56,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010048_5144576.pth [2023-09-19 10:38:56,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010504_5378048.pth [2023-09-19 10:39:01,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 6886.8). Total num frames: 10760192. Throughput: 0: 3459.1, 1: 3458.4. Samples: 10526252. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:39:01,682][45164] Avg episode reward: [(0, '4648.683'), (1, '3256.015')] [2023-09-19 10:39:06,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6886.8). Total num frames: 10792960. Throughput: 0: 3434.7, 1: 3434.8. Samples: 10547370. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:39:06,683][45164] Avg episode reward: [(0, '4945.040'), (1, '3301.198')] [2023-09-19 10:39:06,829][45898] Updated weights for policy 1, policy_version 10320 (0.0013) [2023-09-19 10:39:06,829][45899] Updated weights for policy 0, policy_version 10776 (0.0013) [2023-09-19 10:39:11,682][45164] Fps is (10 sec: 7372.6, 60 sec: 7099.7, 300 sec: 6914.6). Total num frames: 10833920. Throughput: 0: 3485.6, 1: 3486.6. Samples: 10594376. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:39:11,683][45164] Avg episode reward: [(0, '5244.319'), (1, '3523.884')] [2023-09-19 10:39:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010808_5533696.pth... [2023-09-19 10:39:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010352_5300224.pth... [2023-09-19 10:39:11,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010144_5193728.pth [2023-09-19 10:39:11,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010600_5427200.pth [2023-09-19 10:39:16,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6886.8). Total num frames: 10866688. Throughput: 0: 3524.4, 1: 3524.5. Samples: 10637914. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:39:16,682][45164] Avg episode reward: [(0, '5252.221'), (1, '3825.137')] [2023-09-19 10:39:16,709][45811] Saving new best policy, reward=3825.137! [2023-09-19 10:39:18,073][45899] Updated weights for policy 0, policy_version 10856 (0.0014) [2023-09-19 10:39:18,073][45898] Updated weights for policy 1, policy_version 10400 (0.0013) [2023-09-19 10:39:21,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 10907648. Throughput: 0: 3550.5, 1: 3549.9. Samples: 10657858. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:39:21,682][45164] Avg episode reward: [(0, '5015.906'), (1, '4004.802')] [2023-09-19 10:39:21,683][45811] Saving new best policy, reward=4004.802! [2023-09-19 10:39:26,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 10940416. Throughput: 0: 3519.1, 1: 3518.2. Samples: 10698290. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:39:26,683][45164] Avg episode reward: [(0, '4660.835'), (1, '4254.723')] [2023-09-19 10:39:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010456_5353472.pth... [2023-09-19 10:39:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010912_5586944.pth... [2023-09-19 10:39:26,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010240_5242880.pth [2023-09-19 10:39:26,700][45811] Saving new best policy, reward=4254.723! [2023-09-19 10:39:26,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010696_5476352.pth [2023-09-19 10:39:29,611][45899] Updated weights for policy 0, policy_version 10936 (0.0014) [2023-09-19 10:39:29,611][45898] Updated weights for policy 1, policy_version 10480 (0.0014) [2023-09-19 10:39:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 10973184. Throughput: 0: 3502.5, 1: 3502.4. Samples: 10740938. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:39:31,682][45164] Avg episode reward: [(0, '4608.214'), (1, '4199.475')] [2023-09-19 10:39:36,681][45164] Fps is (10 sec: 6553.8, 60 sec: 7099.7, 300 sec: 6914.6). Total num frames: 11005952. Throughput: 0: 3551.2, 1: 3550.9. Samples: 10760810. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:39:36,682][45164] Avg episode reward: [(0, '4561.889'), (1, '4327.738')] [2023-09-19 10:39:36,682][45811] Saving new best policy, reward=4327.738! [2023-09-19 10:39:41,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 11038720. Throughput: 0: 3528.8, 1: 3527.5. Samples: 10797056. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:39:41,682][45164] Avg episode reward: [(0, '4666.165'), (1, '4417.675')] [2023-09-19 10:39:41,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011008_5636096.pth... [2023-09-19 10:39:41,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010552_5402624.pth... [2023-09-19 10:39:41,695][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010808_5533696.pth [2023-09-19 10:39:41,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010352_5300224.pth [2023-09-19 10:39:41,700][45811] Saving new best policy, reward=4417.675! [2023-09-19 10:39:42,431][45899] Updated weights for policy 0, policy_version 11016 (0.0013) [2023-09-19 10:39:42,432][45898] Updated weights for policy 1, policy_version 10560 (0.0013) [2023-09-19 10:39:46,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 11071488. Throughput: 0: 3489.7, 1: 3490.3. Samples: 10840352. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:39:46,683][45164] Avg episode reward: [(0, '4886.729'), (1, '4436.546')] [2023-09-19 10:39:46,684][45811] Saving new best policy, reward=4436.546! [2023-09-19 10:39:51,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 11112448. Throughput: 0: 3503.2, 1: 3501.8. Samples: 10862594. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:39:51,682][45164] Avg episode reward: [(0, '5048.576'), (1, '4473.733')] [2023-09-19 10:39:51,683][45811] Saving new best policy, reward=4473.733! [2023-09-19 10:39:53,562][45899] Updated weights for policy 0, policy_version 11096 (0.0013) [2023-09-19 10:39:53,562][45898] Updated weights for policy 1, policy_version 10640 (0.0015) [2023-09-19 10:39:56,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 11145216. Throughput: 0: 3483.2, 1: 3483.2. Samples: 10907866. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:39:56,683][45164] Avg episode reward: [(0, '5008.082'), (1, '4422.132')] [2023-09-19 10:39:56,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011112_5689344.pth... [2023-09-19 10:39:56,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010656_5455872.pth... [2023-09-19 10:39:56,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010456_5353472.pth [2023-09-19 10:39:56,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000010912_5586944.pth [2023-09-19 10:40:01,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 11186176. Throughput: 0: 3495.5, 1: 3494.4. Samples: 10952460. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:40:01,682][45164] Avg episode reward: [(0, '5055.098'), (1, '4461.859')] [2023-09-19 10:40:04,529][45898] Updated weights for policy 1, policy_version 10720 (0.0013) [2023-09-19 10:40:04,530][45899] Updated weights for policy 0, policy_version 11176 (0.0013) [2023-09-19 10:40:06,681][45164] Fps is (10 sec: 8192.2, 60 sec: 7236.3, 300 sec: 6970.1). Total num frames: 11227136. Throughput: 0: 3519.0, 1: 3520.4. Samples: 10974630. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:40:06,682][45164] Avg episode reward: [(0, '5110.225'), (1, '4411.809')] [2023-09-19 10:40:11,682][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 11259904. Throughput: 0: 3576.4, 1: 3577.3. Samples: 11020210. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:40:11,683][45164] Avg episode reward: [(0, '5141.335'), (1, '4341.187')] [2023-09-19 10:40:11,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010768_5513216.pth... [2023-09-19 10:40:11,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011224_5746688.pth... [2023-09-19 10:40:11,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011008_5636096.pth [2023-09-19 10:40:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010552_5402624.pth [2023-09-19 10:40:15,900][45899] Updated weights for policy 0, policy_version 11256 (0.0013) [2023-09-19 10:40:15,901][45898] Updated weights for policy 1, policy_version 10800 (0.0013) [2023-09-19 10:40:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 11292672. Throughput: 0: 3557.8, 1: 3558.0. Samples: 11061146. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:40:16,682][45164] Avg episode reward: [(0, '4911.545'), (1, '4616.016')] [2023-09-19 10:40:16,683][45811] Saving new best policy, reward=4616.016! [2023-09-19 10:40:21,681][45164] Fps is (10 sec: 7373.0, 60 sec: 7099.7, 300 sec: 6997.9). Total num frames: 11333632. Throughput: 0: 3581.4, 1: 3581.6. Samples: 11083146. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:40:21,682][45164] Avg episode reward: [(0, '5139.095'), (1, '4674.902')] [2023-09-19 10:40:21,683][45811] Saving new best policy, reward=4674.902! [2023-09-19 10:40:26,682][45164] Fps is (10 sec: 6553.4, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 11358208. Throughput: 0: 3598.5, 1: 3599.5. Samples: 11120970. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:40:26,683][45164] Avg episode reward: [(0, '5219.483'), (1, '4685.480')] [2023-09-19 10:40:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010864_5562368.pth... [2023-09-19 10:40:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011320_5795840.pth... [2023-09-19 10:40:26,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010656_5455872.pth [2023-09-19 10:40:26,699][45811] Saving new best policy, reward=4685.480! [2023-09-19 10:40:26,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011112_5689344.pth [2023-09-19 10:40:28,058][45899] Updated weights for policy 0, policy_version 11336 (0.0016) [2023-09-19 10:40:28,058][45898] Updated weights for policy 1, policy_version 10880 (0.0013) [2023-09-19 10:40:31,681][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 11399168. Throughput: 0: 3605.4, 1: 3605.3. Samples: 11164836. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:40:31,682][45164] Avg episode reward: [(0, '5430.138'), (1, '4736.536')] [2023-09-19 10:40:31,684][45811] Saving new best policy, reward=4736.536! [2023-09-19 10:40:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 11431936. Throughput: 0: 3581.2, 1: 3582.5. Samples: 11184964. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:40:36,683][45164] Avg episode reward: [(0, '5610.430'), (1, '4687.887')] [2023-09-19 10:40:39,349][45898] Updated weights for policy 1, policy_version 10960 (0.0012) [2023-09-19 10:40:39,349][45899] Updated weights for policy 0, policy_version 11416 (0.0016) [2023-09-19 10:40:41,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7099.7, 300 sec: 6942.4). Total num frames: 11464704. Throughput: 0: 3572.4, 1: 3572.4. Samples: 11229380. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:40:41,682][45164] Avg episode reward: [(0, '5755.923'), (1, '4383.870')] [2023-09-19 10:40:41,686][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010976_5619712.pth... [2023-09-19 10:40:41,689][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010768_5513216.pth [2023-09-19 10:40:41,696][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011432_5853184.pth... [2023-09-19 10:40:41,704][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011224_5746688.pth [2023-09-19 10:40:41,704][45810] Saving new best policy, reward=5755.923! [2023-09-19 10:40:46,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7099.8, 300 sec: 6914.6). Total num frames: 11497472. Throughput: 0: 3492.1, 1: 3493.0. Samples: 11266788. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:40:46,682][45164] Avg episode reward: [(0, '5541.689'), (1, '4437.660')] [2023-09-19 10:40:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 11530240. Throughput: 0: 3464.8, 1: 3464.7. Samples: 11286454. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:40:51,682][45164] Avg episode reward: [(0, '5349.056'), (1, '4412.231')] [2023-09-19 10:40:52,085][45899] Updated weights for policy 0, policy_version 11496 (0.0012) [2023-09-19 10:40:52,086][45898] Updated weights for policy 1, policy_version 11040 (0.0015) [2023-09-19 10:40:56,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 11563008. Throughput: 0: 3373.6, 1: 3373.6. Samples: 11323834. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:40:56,682][45164] Avg episode reward: [(0, '4879.892'), (1, '4349.569')] [2023-09-19 10:40:56,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011064_5664768.pth... [2023-09-19 10:40:56,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011520_5898240.pth... [2023-09-19 10:40:56,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010864_5562368.pth [2023-09-19 10:40:56,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011320_5795840.pth [2023-09-19 10:41:01,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 11595776. Throughput: 0: 3413.9, 1: 3414.1. Samples: 11368406. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:41:01,683][45164] Avg episode reward: [(0, '4841.947'), (1, '4292.823')] [2023-09-19 10:41:03,994][45898] Updated weights for policy 1, policy_version 11120 (0.0015) [2023-09-19 10:41:03,994][45899] Updated weights for policy 0, policy_version 11576 (0.0014) [2023-09-19 10:41:06,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 11636736. Throughput: 0: 3386.5, 1: 3386.5. Samples: 11387934. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:41:06,682][45164] Avg episode reward: [(0, '4911.540'), (1, '4332.728')] [2023-09-19 10:41:11,682][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 11669504. Throughput: 0: 3446.6, 1: 3446.8. Samples: 11431170. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:41:11,683][45164] Avg episode reward: [(0, '4723.927'), (1, '4309.165')] [2023-09-19 10:41:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011624_5951488.pth... [2023-09-19 10:41:11,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011168_5718016.pth... [2023-09-19 10:41:11,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000010976_5619712.pth [2023-09-19 10:41:11,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011432_5853184.pth [2023-09-19 10:41:16,069][45898] Updated weights for policy 1, policy_version 11200 (0.0011) [2023-09-19 10:41:16,070][45899] Updated weights for policy 0, policy_version 11656 (0.0014) [2023-09-19 10:41:16,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 11702272. Throughput: 0: 3382.4, 1: 3382.6. Samples: 11469260. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:41:16,682][45164] Avg episode reward: [(0, '4932.045'), (1, '4137.948')] [2023-09-19 10:41:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6690.1, 300 sec: 6914.6). Total num frames: 11735040. Throughput: 0: 3366.0, 1: 3365.8. Samples: 11487892. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:41:21,682][45164] Avg episode reward: [(0, '5034.986'), (1, '3983.175')] [2023-09-19 10:41:26,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6690.2, 300 sec: 6859.1). Total num frames: 11759616. Throughput: 0: 3279.2, 1: 3279.2. Samples: 11524508. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:41:26,682][45164] Avg episode reward: [(0, '5144.403'), (1, '3765.275')] [2023-09-19 10:41:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011712_5996544.pth... [2023-09-19 10:41:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011256_5763072.pth... [2023-09-19 10:41:26,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011064_5664768.pth [2023-09-19 10:41:26,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011520_5898240.pth [2023-09-19 10:41:29,062][45899] Updated weights for policy 0, policy_version 11736 (0.0010) [2023-09-19 10:41:29,063][45898] Updated weights for policy 1, policy_version 11280 (0.0014) [2023-09-19 10:41:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6886.8). Total num frames: 11800576. Throughput: 0: 3304.5, 1: 3304.4. Samples: 11564186. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:41:31,682][45164] Avg episode reward: [(0, '5154.020'), (1, '3538.890')] [2023-09-19 10:41:36,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6690.1, 300 sec: 6886.8). Total num frames: 11833344. Throughput: 0: 3321.1, 1: 3321.1. Samples: 11585354. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:41:36,683][45164] Avg episode reward: [(0, '5144.634'), (1, '3598.821')] [2023-09-19 10:41:41,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 11857920. Throughput: 0: 3311.3, 1: 3310.9. Samples: 11621832. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:41:41,682][45164] Avg episode reward: [(0, '5201.049'), (1, '3769.433')] [2023-09-19 10:41:41,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011808_6045696.pth... [2023-09-19 10:41:41,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011352_5812224.pth... [2023-09-19 10:41:41,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011168_5718016.pth [2023-09-19 10:41:41,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011624_5951488.pth [2023-09-19 10:41:42,494][45899] Updated weights for policy 0, policy_version 11816 (0.0016) [2023-09-19 10:41:42,494][45898] Updated weights for policy 1, policy_version 11360 (0.0015) [2023-09-19 10:41:46,681][45164] Fps is (10 sec: 5734.5, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 11890688. Throughput: 0: 3198.4, 1: 3198.3. Samples: 11656256. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:41:46,682][45164] Avg episode reward: [(0, '5205.593'), (1, '3783.497')] [2023-09-19 10:41:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 11923456. Throughput: 0: 3171.5, 1: 3170.8. Samples: 11673336. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:41:51,683][45164] Avg episode reward: [(0, '5112.326'), (1, '3864.928')] [2023-09-19 10:41:55,240][45899] Updated weights for policy 0, policy_version 11896 (0.0014) [2023-09-19 10:41:55,241][45898] Updated weights for policy 1, policy_version 11440 (0.0014) [2023-09-19 10:41:56,682][45164] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 11956224. Throughput: 0: 3148.8, 1: 3147.8. Samples: 11714514. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:41:56,683][45164] Avg episode reward: [(0, '5194.384'), (1, '4064.055')] [2023-09-19 10:41:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011448_5861376.pth... [2023-09-19 10:41:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011904_6094848.pth... [2023-09-19 10:41:56,695][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011256_5763072.pth [2023-09-19 10:41:56,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011712_5996544.pth [2023-09-19 10:42:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 11988992. Throughput: 0: 3177.9, 1: 3177.7. Samples: 11755260. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:42:01,683][45164] Avg episode reward: [(0, '5208.623'), (1, '4037.658')] [2023-09-19 10:42:06,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6831.3). Total num frames: 12021760. Throughput: 0: 3163.6, 1: 3163.7. Samples: 11772620. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:42:06,682][45164] Avg episode reward: [(0, '5314.442'), (1, '4130.144')] [2023-09-19 10:42:07,275][45898] Updated weights for policy 1, policy_version 11520 (0.0014) [2023-09-19 10:42:07,275][45899] Updated weights for policy 0, policy_version 11976 (0.0012) [2023-09-19 10:42:11,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 12062720. Throughput: 0: 3263.1, 1: 3263.1. Samples: 11818188. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:42:11,682][45164] Avg episode reward: [(0, '5411.190'), (1, '4208.542')] [2023-09-19 10:42:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011552_5914624.pth... [2023-09-19 10:42:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012008_6148096.pth... [2023-09-19 10:42:11,695][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011352_5812224.pth [2023-09-19 10:42:11,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011808_6045696.pth [2023-09-19 10:42:16,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 12095488. Throughput: 0: 3272.5, 1: 3272.7. Samples: 11858720. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:42:16,683][45164] Avg episode reward: [(0, '5496.072'), (1, '4152.725')] [2023-09-19 10:42:19,135][45898] Updated weights for policy 1, policy_version 11600 (0.0010) [2023-09-19 10:42:19,137][45899] Updated weights for policy 0, policy_version 12056 (0.0015) [2023-09-19 10:42:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 12128256. Throughput: 0: 3257.4, 1: 3256.1. Samples: 11878464. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:42:21,682][45164] Avg episode reward: [(0, '5618.589'), (1, '4165.427')] [2023-09-19 10:42:26,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 12161024. Throughput: 0: 3306.3, 1: 3306.1. Samples: 11919394. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:42:26,683][45164] Avg episode reward: [(0, '5683.663'), (1, '4258.944')] [2023-09-19 10:42:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012104_6197248.pth... [2023-09-19 10:42:26,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011648_5963776.pth... [2023-09-19 10:42:26,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011448_5861376.pth [2023-09-19 10:42:26,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000011904_6094848.pth [2023-09-19 10:42:30,889][45898] Updated weights for policy 1, policy_version 11680 (0.0012) [2023-09-19 10:42:30,889][45899] Updated weights for policy 0, policy_version 12136 (0.0013) [2023-09-19 10:42:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 12193792. Throughput: 0: 3400.4, 1: 3400.4. Samples: 11962294. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:42:31,683][45164] Avg episode reward: [(0, '5564.820'), (1, '4315.025')] [2023-09-19 10:42:36,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 12226560. Throughput: 0: 3428.6, 1: 3429.4. Samples: 11981944. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:42:36,682][45164] Avg episode reward: [(0, '5548.501'), (1, '4337.830')] [2023-09-19 10:42:41,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6831.3). Total num frames: 12259328. Throughput: 0: 3370.5, 1: 3371.3. Samples: 12017894. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:42:41,682][45164] Avg episode reward: [(0, '5605.783'), (1, '4329.865')] [2023-09-19 10:42:41,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011744_6012928.pth... [2023-09-19 10:42:41,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012200_6246400.pth... [2023-09-19 10:42:41,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011552_5914624.pth [2023-09-19 10:42:41,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012008_6148096.pth [2023-09-19 10:42:44,474][45899] Updated weights for policy 0, policy_version 12216 (0.0013) [2023-09-19 10:42:44,474][45898] Updated weights for policy 1, policy_version 11760 (0.0013) [2023-09-19 10:42:46,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 12283904. Throughput: 0: 3295.1, 1: 3295.0. Samples: 12051814. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:42:46,682][45164] Avg episode reward: [(0, '5613.498'), (1, '4176.824')] [2023-09-19 10:42:51,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 12316672. Throughput: 0: 3349.7, 1: 3349.7. Samples: 12074096. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:42:51,683][45164] Avg episode reward: [(0, '5671.750'), (1, '4093.623')] [2023-09-19 10:42:56,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 12349440. Throughput: 0: 3240.6, 1: 3240.5. Samples: 12109836. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:42:56,682][45164] Avg episode reward: [(0, '5774.082'), (1, '3951.416')] [2023-09-19 10:42:56,687][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012288_6291456.pth... [2023-09-19 10:42:56,687][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011832_6057984.pth... [2023-09-19 10:42:56,692][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012104_6197248.pth [2023-09-19 10:42:56,693][45810] Saving new best policy, reward=5774.082! [2023-09-19 10:42:56,693][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011648_5963776.pth [2023-09-19 10:42:57,648][45899] Updated weights for policy 0, policy_version 12296 (0.0013) [2023-09-19 10:42:57,649][45898] Updated weights for policy 1, policy_version 11840 (0.0012) [2023-09-19 10:43:01,682][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6803.5). Total num frames: 12382208. Throughput: 0: 3209.6, 1: 3209.5. Samples: 12147582. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:43:01,683][45164] Avg episode reward: [(0, '5784.669'), (1, '3996.182')] [2023-09-19 10:43:01,684][45810] Saving new best policy, reward=5784.669! [2023-09-19 10:43:06,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6803.5). Total num frames: 12414976. Throughput: 0: 3223.2, 1: 3224.1. Samples: 12168592. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:43:06,683][45164] Avg episode reward: [(0, '5766.821'), (1, '4088.381')] [2023-09-19 10:43:10,185][45898] Updated weights for policy 1, policy_version 11920 (0.0015) [2023-09-19 10:43:10,186][45899] Updated weights for policy 0, policy_version 12376 (0.0012) [2023-09-19 10:43:11,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6775.8). Total num frames: 12447744. Throughput: 0: 3185.7, 1: 3185.1. Samples: 12206082. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:43:11,682][45164] Avg episode reward: [(0, '5579.428'), (1, '4037.668')] [2023-09-19 10:43:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011928_6107136.pth... [2023-09-19 10:43:11,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012384_6340608.pth... [2023-09-19 10:43:11,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011744_6012928.pth [2023-09-19 10:43:11,704][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012200_6246400.pth [2023-09-19 10:43:16,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6417.0, 300 sec: 6748.0). Total num frames: 12480512. Throughput: 0: 3165.1, 1: 3164.0. Samples: 12247106. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:43:16,683][45164] Avg episode reward: [(0, '5673.799'), (1, '4258.591')] [2023-09-19 10:43:21,482][45899] Updated weights for policy 0, policy_version 12456 (0.0015) [2023-09-19 10:43:21,482][45898] Updated weights for policy 1, policy_version 12000 (0.0011) [2023-09-19 10:43:21,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6553.6, 300 sec: 6775.8). Total num frames: 12521472. Throughput: 0: 3210.1, 1: 3210.2. Samples: 12270856. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:43:21,682][45164] Avg episode reward: [(0, '5607.611'), (1, '4254.870')] [2023-09-19 10:43:26,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6748.0). Total num frames: 12546048. Throughput: 0: 3247.0, 1: 3247.2. Samples: 12310134. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:43:26,683][45164] Avg episode reward: [(0, '5602.645'), (1, '4099.901')] [2023-09-19 10:43:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012480_6389760.pth... [2023-09-19 10:43:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012024_6156288.pth... [2023-09-19 10:43:26,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012288_6291456.pth [2023-09-19 10:43:26,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011832_6057984.pth [2023-09-19 10:43:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6803.5). Total num frames: 12587008. Throughput: 0: 3314.0, 1: 3314.3. Samples: 12350090. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:43:31,683][45164] Avg episode reward: [(0, '5635.462'), (1, '4152.617')] [2023-09-19 10:43:33,895][45899] Updated weights for policy 0, policy_version 12536 (0.0012) [2023-09-19 10:43:33,895][45898] Updated weights for policy 1, policy_version 12080 (0.0015) [2023-09-19 10:43:36,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6553.6, 300 sec: 6775.8). Total num frames: 12619776. Throughput: 0: 3309.0, 1: 3309.0. Samples: 12371906. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:43:36,682][45164] Avg episode reward: [(0, '5816.710'), (1, '4102.731')] [2023-09-19 10:43:36,684][45810] Saving new best policy, reward=5816.710! [2023-09-19 10:43:41,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6775.8). Total num frames: 12652544. Throughput: 0: 3382.7, 1: 3382.7. Samples: 12414280. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:43:41,682][45164] Avg episode reward: [(0, '5752.769'), (1, '4064.798')] [2023-09-19 10:43:41,688][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012584_6443008.pth... [2023-09-19 10:43:41,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012128_6209536.pth... [2023-09-19 10:43:41,696][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012384_6340608.pth [2023-09-19 10:43:41,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000011928_6107136.pth [2023-09-19 10:43:46,014][45898] Updated weights for policy 1, policy_version 12160 (0.0014) [2023-09-19 10:43:46,014][45899] Updated weights for policy 0, policy_version 12616 (0.0015) [2023-09-19 10:43:46,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6775.8). Total num frames: 12685312. Throughput: 0: 3390.3, 1: 3390.3. Samples: 12452708. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:43:46,682][45164] Avg episode reward: [(0, '5797.086'), (1, '3856.994')] [2023-09-19 10:43:51,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6803.5). Total num frames: 12726272. Throughput: 0: 3420.7, 1: 3419.8. Samples: 12476416. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:43:51,683][45164] Avg episode reward: [(0, '6034.009'), (1, '3275.179')] [2023-09-19 10:43:51,684][45810] Saving new best policy, reward=6034.009! [2023-09-19 10:43:56,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.6, 300 sec: 6775.8). Total num frames: 12759040. Throughput: 0: 3451.0, 1: 3451.5. Samples: 12516694. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:43:56,683][45164] Avg episode reward: [(0, '6128.291'), (1, '2741.201')] [2023-09-19 10:43:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012232_6262784.pth... [2023-09-19 10:43:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012688_6496256.pth... [2023-09-19 10:43:56,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012024_6156288.pth [2023-09-19 10:43:56,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012480_6389760.pth [2023-09-19 10:43:56,700][45810] Saving new best policy, reward=6128.291! [2023-09-19 10:43:57,616][45899] Updated weights for policy 0, policy_version 12696 (0.0011) [2023-09-19 10:43:57,617][45898] Updated weights for policy 1, policy_version 12240 (0.0015) [2023-09-19 10:44:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6775.8). Total num frames: 12791808. Throughput: 0: 3457.0, 1: 3457.2. Samples: 12558244. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:44:01,682][45164] Avg episode reward: [(0, '6095.938'), (1, '2800.146')] [2023-09-19 10:44:06,681][45164] Fps is (10 sec: 5734.5, 60 sec: 6690.1, 300 sec: 6720.2). Total num frames: 12816384. Throughput: 0: 3384.1, 1: 3383.9. Samples: 12575416. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:06,682][45164] Avg episode reward: [(0, '6064.160'), (1, '2965.230')] [2023-09-19 10:44:09,905][45899] Updated weights for policy 0, policy_version 12776 (0.0011) [2023-09-19 10:44:09,905][45898] Updated weights for policy 1, policy_version 12320 (0.0010) [2023-09-19 10:44:11,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6748.0). Total num frames: 12857344. Throughput: 0: 3415.7, 1: 3415.7. Samples: 12617550. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:11,682][45164] Avg episode reward: [(0, '6113.234'), (1, '3352.653')] [2023-09-19 10:44:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012784_6545408.pth... [2023-09-19 10:44:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012328_6311936.pth... [2023-09-19 10:44:11,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012584_6443008.pth [2023-09-19 10:44:11,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012128_6209536.pth [2023-09-19 10:44:16,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 6720.2). Total num frames: 12890112. Throughput: 0: 3449.0, 1: 3448.9. Samples: 12660494. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:16,682][45164] Avg episode reward: [(0, '6085.097'), (1, '3499.044')] [2023-09-19 10:44:21,091][45898] Updated weights for policy 1, policy_version 12400 (0.0013) [2023-09-19 10:44:21,091][45899] Updated weights for policy 0, policy_version 12856 (0.0014) [2023-09-19 10:44:21,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6748.0). Total num frames: 12931072. Throughput: 0: 3454.1, 1: 3453.8. Samples: 12682760. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:21,682][45164] Avg episode reward: [(0, '6034.648'), (1, '3426.525')] [2023-09-19 10:44:26,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6748.0). Total num frames: 12963840. Throughput: 0: 3467.1, 1: 3467.3. Samples: 12726330. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:26,683][45164] Avg episode reward: [(0, '6067.774'), (1, '3370.870')] [2023-09-19 10:44:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012888_6598656.pth... [2023-09-19 10:44:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012432_6365184.pth... [2023-09-19 10:44:26,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012688_6496256.pth [2023-09-19 10:44:26,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012232_6262784.pth [2023-09-19 10:44:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6775.8). Total num frames: 13004800. Throughput: 0: 3535.6, 1: 3535.7. Samples: 12770918. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:31,682][45164] Avg episode reward: [(0, '6127.862'), (1, '3491.181')] [2023-09-19 10:44:32,382][45898] Updated weights for policy 1, policy_version 12480 (0.0014) [2023-09-19 10:44:32,382][45899] Updated weights for policy 0, policy_version 12936 (0.0011) [2023-09-19 10:44:36,682][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6775.8). Total num frames: 13037568. Throughput: 0: 3521.3, 1: 3522.6. Samples: 12793390. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:36,682][45164] Avg episode reward: [(0, '6128.567'), (1, '3709.261')] [2023-09-19 10:44:36,696][45810] Saving new best policy, reward=6128.567! [2023-09-19 10:44:41,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6803.5). Total num frames: 13078528. Throughput: 0: 3578.7, 1: 3579.1. Samples: 12838794. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:41,682][45164] Avg episode reward: [(0, '6110.693'), (1, '3869.955')] [2023-09-19 10:44:41,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013000_6656000.pth... [2023-09-19 10:44:41,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012544_6422528.pth... [2023-09-19 10:44:41,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012328_6311936.pth [2023-09-19 10:44:41,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012784_6545408.pth [2023-09-19 10:44:43,185][45898] Updated weights for policy 1, policy_version 12560 (0.0012) [2023-09-19 10:44:43,186][45899] Updated weights for policy 0, policy_version 13016 (0.0014) [2023-09-19 10:44:46,681][45164] Fps is (10 sec: 8192.3, 60 sec: 7236.3, 300 sec: 6803.5). Total num frames: 13119488. Throughput: 0: 3620.5, 1: 3621.5. Samples: 12884132. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:46,682][45164] Avg episode reward: [(0, '6010.709'), (1, '3915.471')] [2023-09-19 10:44:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.7, 300 sec: 6803.5). Total num frames: 13152256. Throughput: 0: 3664.7, 1: 3664.7. Samples: 12905236. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:44:51,682][45164] Avg episode reward: [(0, '6072.601'), (1, '3885.352')] [2023-09-19 10:44:54,290][45898] Updated weights for policy 1, policy_version 12640 (0.0015) [2023-09-19 10:44:54,290][45899] Updated weights for policy 0, policy_version 13096 (0.0015) [2023-09-19 10:44:56,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7236.3, 300 sec: 6803.5). Total num frames: 13193216. Throughput: 0: 3699.0, 1: 3698.9. Samples: 12950456. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:44:56,682][45164] Avg episode reward: [(0, '6127.793'), (1, '4021.815')] [2023-09-19 10:44:56,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013112_6713344.pth... [2023-09-19 10:44:56,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012656_6479872.pth... [2023-09-19 10:44:56,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000012888_6598656.pth [2023-09-19 10:44:56,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012432_6365184.pth [2023-09-19 10:45:01,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 6775.8). Total num frames: 13225984. Throughput: 0: 3707.2, 1: 3707.2. Samples: 12994138. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:45:01,682][45164] Avg episode reward: [(0, '6120.851'), (1, '4091.901')] [2023-09-19 10:45:05,813][45899] Updated weights for policy 0, policy_version 13176 (0.0015) [2023-09-19 10:45:05,813][45898] Updated weights for policy 1, policy_version 12720 (0.0011) [2023-09-19 10:45:06,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7372.8, 300 sec: 6775.8). Total num frames: 13258752. Throughput: 0: 3678.0, 1: 3678.3. Samples: 13013796. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:45:06,682][45164] Avg episode reward: [(0, '6182.690'), (1, '4118.264')] [2023-09-19 10:45:06,684][45810] Saving new best policy, reward=6182.690! [2023-09-19 10:45:11,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7372.8, 300 sec: 6803.5). Total num frames: 13299712. Throughput: 0: 3679.7, 1: 3679.7. Samples: 13057500. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:45:11,682][45164] Avg episode reward: [(0, '6307.096'), (1, '4281.485')] [2023-09-19 10:45:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013216_6766592.pth... [2023-09-19 10:45:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012760_6533120.pth... [2023-09-19 10:45:11,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013000_6656000.pth [2023-09-19 10:45:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012544_6422528.pth [2023-09-19 10:45:11,701][45810] Saving new best policy, reward=6307.096! [2023-09-19 10:45:16,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7372.8, 300 sec: 6775.8). Total num frames: 13332480. Throughput: 0: 3646.1, 1: 3645.6. Samples: 13099044. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:45:16,682][45164] Avg episode reward: [(0, '6375.887'), (1, '4174.024')] [2023-09-19 10:45:16,683][45810] Saving new best policy, reward=6375.887! [2023-09-19 10:45:17,272][45899] Updated weights for policy 0, policy_version 13256 (0.0016) [2023-09-19 10:45:17,272][45898] Updated weights for policy 1, policy_version 12800 (0.0016) [2023-09-19 10:45:21,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7236.3, 300 sec: 6803.5). Total num frames: 13365248. Throughput: 0: 3632.1, 1: 3632.2. Samples: 13120282. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:45:21,682][45164] Avg episode reward: [(0, '6472.111'), (1, '4242.968')] [2023-09-19 10:45:21,684][45810] Saving new best policy, reward=6472.111! [2023-09-19 10:45:26,682][45164] Fps is (10 sec: 7372.7, 60 sec: 7372.8, 300 sec: 6803.5). Total num frames: 13406208. Throughput: 0: 3616.3, 1: 3616.5. Samples: 13164272. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:45:26,683][45164] Avg episode reward: [(0, '6553.825'), (1, '4253.888')] [2023-09-19 10:45:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012864_6586368.pth... [2023-09-19 10:45:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013320_6819840.pth... [2023-09-19 10:45:26,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012656_6479872.pth [2023-09-19 10:45:26,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013112_6713344.pth [2023-09-19 10:45:26,702][45810] Saving new best policy, reward=6553.825! [2023-09-19 10:45:28,529][45899] Updated weights for policy 0, policy_version 13336 (0.0013) [2023-09-19 10:45:28,529][45898] Updated weights for policy 1, policy_version 12880 (0.0012) [2023-09-19 10:45:31,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7236.3, 300 sec: 6803.5). Total num frames: 13438976. Throughput: 0: 3572.1, 1: 3570.9. Samples: 13205570. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:45:31,682][45164] Avg episode reward: [(0, '6584.619'), (1, '4269.612')] [2023-09-19 10:45:31,683][45810] Saving new best policy, reward=6584.619! [2023-09-19 10:45:36,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7236.3, 300 sec: 6803.5). Total num frames: 13471744. Throughput: 0: 3572.1, 1: 3572.2. Samples: 13226730. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:45:36,682][45164] Avg episode reward: [(0, '6605.894'), (1, '4474.995')] [2023-09-19 10:45:36,683][45810] Saving new best policy, reward=6605.894! [2023-09-19 10:45:40,389][45898] Updated weights for policy 1, policy_version 12960 (0.0015) [2023-09-19 10:45:40,389][45899] Updated weights for policy 0, policy_version 13416 (0.0013) [2023-09-19 10:45:41,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7236.3, 300 sec: 6831.3). Total num frames: 13512704. Throughput: 0: 3547.5, 1: 3547.4. Samples: 13269726. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:45:41,682][45164] Avg episode reward: [(0, '6583.705'), (1, '4584.039')] [2023-09-19 10:45:41,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013424_6873088.pth... [2023-09-19 10:45:41,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012968_6639616.pth... [2023-09-19 10:45:41,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013216_6766592.pth [2023-09-19 10:45:41,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012760_6533120.pth [2023-09-19 10:45:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6831.3). Total num frames: 13545472. Throughput: 0: 3536.6, 1: 3536.7. Samples: 13312434. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:45:46,682][45164] Avg episode reward: [(0, '6550.425'), (1, '4647.961')] [2023-09-19 10:45:51,675][45899] Updated weights for policy 0, policy_version 13496 (0.0012) [2023-09-19 10:45:51,676][45898] Updated weights for policy 1, policy_version 13040 (0.0012) [2023-09-19 10:45:51,683][45164] Fps is (10 sec: 7371.9, 60 sec: 7236.1, 300 sec: 6859.0). Total num frames: 13586432. Throughput: 0: 3561.1, 1: 3561.1. Samples: 13334302. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:45:51,685][45164] Avg episode reward: [(0, '6543.252'), (1, '4439.626')] [2023-09-19 10:45:56,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6831.3). Total num frames: 13611008. Throughput: 0: 3556.6, 1: 3555.3. Samples: 13377536. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:45:56,682][45164] Avg episode reward: [(0, '6595.809'), (1, '4282.551')] [2023-09-19 10:45:56,722][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013072_6692864.pth... [2023-09-19 10:45:56,727][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012864_6586368.pth [2023-09-19 10:45:56,731][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013528_6926336.pth... [2023-09-19 10:45:56,735][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013320_6819840.pth [2023-09-19 10:46:01,681][45164] Fps is (10 sec: 5735.2, 60 sec: 6963.2, 300 sec: 6803.5). Total num frames: 13643776. Throughput: 0: 3500.0, 1: 3500.7. Samples: 13414074. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:46:01,682][45164] Avg episode reward: [(0, '6425.922'), (1, '3863.235')] [2023-09-19 10:46:04,225][45899] Updated weights for policy 0, policy_version 13576 (0.0013) [2023-09-19 10:46:04,225][45898] Updated weights for policy 1, policy_version 13120 (0.0011) [2023-09-19 10:46:06,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6831.3). Total num frames: 13684736. Throughput: 0: 3496.0, 1: 3494.8. Samples: 13434868. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:46:06,682][45164] Avg episode reward: [(0, '6436.864'), (1, '3806.744')] [2023-09-19 10:46:11,682][45164] Fps is (10 sec: 7372.6, 60 sec: 6963.2, 300 sec: 6831.3). Total num frames: 13717504. Throughput: 0: 3463.2, 1: 3462.0. Samples: 13475906. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:46:11,683][45164] Avg episode reward: [(0, '6415.592'), (1, '3798.831')] [2023-09-19 10:46:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013624_6975488.pth... [2023-09-19 10:46:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013168_6742016.pth... [2023-09-19 10:46:11,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000012968_6639616.pth [2023-09-19 10:46:11,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013424_6873088.pth [2023-09-19 10:46:16,681][45164] Fps is (10 sec: 5734.5, 60 sec: 6826.7, 300 sec: 6803.5). Total num frames: 13742080. Throughput: 0: 3393.5, 1: 3394.5. Samples: 13511032. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:46:16,682][45164] Avg episode reward: [(0, '6424.321'), (1, '3761.684')] [2023-09-19 10:46:17,184][45899] Updated weights for policy 0, policy_version 13656 (0.0010) [2023-09-19 10:46:17,184][45898] Updated weights for policy 1, policy_version 13200 (0.0014) [2023-09-19 10:46:21,681][45164] Fps is (10 sec: 5734.5, 60 sec: 6826.7, 300 sec: 6831.3). Total num frames: 13774848. Throughput: 0: 3339.5, 1: 3339.4. Samples: 13527280. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:46:21,682][45164] Avg episode reward: [(0, '6426.276'), (1, '3928.563')] [2023-09-19 10:46:26,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6553.6, 300 sec: 6775.8). Total num frames: 13799424. Throughput: 0: 3282.4, 1: 3281.6. Samples: 13565104. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:46:26,682][45164] Avg episode reward: [(0, '6405.538'), (1, '3976.190')] [2023-09-19 10:46:26,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013704_7016448.pth... [2023-09-19 10:46:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013248_6782976.pth... [2023-09-19 10:46:26,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013528_6926336.pth [2023-09-19 10:46:26,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013072_6692864.pth [2023-09-19 10:46:30,181][45899] Updated weights for policy 0, policy_version 13736 (0.0015) [2023-09-19 10:46:30,181][45898] Updated weights for policy 1, policy_version 13280 (0.0014) [2023-09-19 10:46:31,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6553.6, 300 sec: 6775.8). Total num frames: 13832192. Throughput: 0: 3253.9, 1: 3253.5. Samples: 13605266. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:46:31,682][45164] Avg episode reward: [(0, '6508.145'), (1, '4055.867')] [2023-09-19 10:46:36,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6803.5). Total num frames: 13864960. Throughput: 0: 3195.2, 1: 3195.2. Samples: 13621866. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:46:36,682][45164] Avg episode reward: [(0, '6559.420'), (1, '4078.296')] [2023-09-19 10:46:41,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 13905920. Throughput: 0: 3207.6, 1: 3208.6. Samples: 13666270. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:46:41,683][45164] Avg episode reward: [(0, '6410.600'), (1, '4266.450')] [2023-09-19 10:46:41,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013808_7069696.pth... [2023-09-19 10:46:41,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013352_6836224.pth... [2023-09-19 10:46:41,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013624_6975488.pth [2023-09-19 10:46:41,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013168_6742016.pth [2023-09-19 10:46:42,150][45898] Updated weights for policy 1, policy_version 13360 (0.0013) [2023-09-19 10:46:42,151][45899] Updated weights for policy 0, policy_version 13816 (0.0016) [2023-09-19 10:46:46,681][45164] Fps is (10 sec: 8192.1, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 13946880. Throughput: 0: 3301.6, 1: 3301.7. Samples: 13711226. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:46:46,682][45164] Avg episode reward: [(0, '6327.363'), (1, '4531.423')] [2023-09-19 10:46:51,681][45164] Fps is (10 sec: 7373.0, 60 sec: 6553.7, 300 sec: 6859.1). Total num frames: 13979648. Throughput: 0: 3324.9, 1: 3325.9. Samples: 13734150. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:46:51,682][45164] Avg episode reward: [(0, '6396.024'), (1, '4626.367')] [2023-09-19 10:46:54,388][45899] Updated weights for policy 0, policy_version 13896 (0.0011) [2023-09-19 10:46:54,388][45898] Updated weights for policy 1, policy_version 13440 (0.0011) [2023-09-19 10:46:56,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 14004224. Throughput: 0: 3252.8, 1: 3254.0. Samples: 13768710. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:46:56,682][45164] Avg episode reward: [(0, '6404.175'), (1, '4472.991')] [2023-09-19 10:46:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013904_7118848.pth... [2023-09-19 10:46:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013448_6885376.pth... [2023-09-19 10:46:56,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013704_7016448.pth [2023-09-19 10:46:56,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013248_6782976.pth [2023-09-19 10:47:01,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 14036992. Throughput: 0: 3243.1, 1: 3242.5. Samples: 13802886. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:47:01,683][45164] Avg episode reward: [(0, '6416.559'), (1, '4485.688')] [2023-09-19 10:47:06,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6280.5, 300 sec: 6775.8). Total num frames: 14061568. Throughput: 0: 3240.8, 1: 3240.4. Samples: 13818936. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:47:06,682][45164] Avg episode reward: [(0, '6303.965'), (1, '4400.730')] [2023-09-19 10:47:08,354][45899] Updated weights for policy 0, policy_version 13976 (0.0013) [2023-09-19 10:47:08,354][45898] Updated weights for policy 1, policy_version 13520 (0.0014) [2023-09-19 10:47:11,682][45164] Fps is (10 sec: 5734.3, 60 sec: 6280.5, 300 sec: 6775.8). Total num frames: 14094336. Throughput: 0: 3257.0, 1: 3257.9. Samples: 13858276. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:47:11,683][45164] Avg episode reward: [(0, '6584.331'), (1, '4397.085')] [2023-09-19 10:47:11,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013536_6930432.pth... [2023-09-19 10:47:11,694][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013992_7163904.pth... [2023-09-19 10:47:11,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013352_6836224.pth [2023-09-19 10:47:11,704][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013808_7069696.pth [2023-09-19 10:47:16,682][45164] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6775.8). Total num frames: 14127104. Throughput: 0: 3236.5, 1: 3236.7. Samples: 13896564. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:47:16,683][45164] Avg episode reward: [(0, '6552.493'), (1, '4434.456')] [2023-09-19 10:47:20,263][45899] Updated weights for policy 0, policy_version 14056 (0.0012) [2023-09-19 10:47:20,264][45898] Updated weights for policy 1, policy_version 13600 (0.0013) [2023-09-19 10:47:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6775.8). Total num frames: 14159872. Throughput: 0: 3294.1, 1: 3293.2. Samples: 13918292. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:47:21,682][45164] Avg episode reward: [(0, '6574.161'), (1, '4715.444')] [2023-09-19 10:47:26,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6690.1, 300 sec: 6803.5). Total num frames: 14200832. Throughput: 0: 3246.8, 1: 3247.0. Samples: 13958492. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:47:26,682][45164] Avg episode reward: [(0, '6588.747'), (1, '4675.235')] [2023-09-19 10:47:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013640_6983680.pth... [2023-09-19 10:47:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014096_7217152.pth... [2023-09-19 10:47:26,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013448_6885376.pth [2023-09-19 10:47:26,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013904_7118848.pth [2023-09-19 10:47:31,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6690.1, 300 sec: 6803.5). Total num frames: 14233600. Throughput: 0: 3213.2, 1: 3212.9. Samples: 14000404. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:47:31,682][45164] Avg episode reward: [(0, '6611.527'), (1, '4729.787')] [2023-09-19 10:47:31,683][45810] Saving new best policy, reward=6611.527! [2023-09-19 10:47:32,331][45899] Updated weights for policy 0, policy_version 14136 (0.0014) [2023-09-19 10:47:32,332][45898] Updated weights for policy 1, policy_version 13680 (0.0014) [2023-09-19 10:47:36,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6803.5). Total num frames: 14266368. Throughput: 0: 3152.9, 1: 3152.8. Samples: 14017904. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:47:36,682][45164] Avg episode reward: [(0, '6621.026'), (1, '4632.707')] [2023-09-19 10:47:36,683][45810] Saving new best policy, reward=6621.026! [2023-09-19 10:47:41,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 14299136. Throughput: 0: 3238.1, 1: 3238.1. Samples: 14060138. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:47:41,683][45164] Avg episode reward: [(0, '6514.607'), (1, '4578.541')] [2023-09-19 10:47:41,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014192_7266304.pth... [2023-09-19 10:47:41,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013736_7032832.pth... [2023-09-19 10:47:41,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000013992_7163904.pth [2023-09-19 10:47:41,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013536_6930432.pth [2023-09-19 10:47:44,396][45899] Updated weights for policy 0, policy_version 14216 (0.0012) [2023-09-19 10:47:44,396][45898] Updated weights for policy 1, policy_version 13760 (0.0013) [2023-09-19 10:47:46,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 14340096. Throughput: 0: 3340.8, 1: 3341.6. Samples: 14103590. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:47:46,683][45164] Avg episode reward: [(0, '6514.704'), (1, '4606.688')] [2023-09-19 10:47:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6859.1). Total num frames: 14372864. Throughput: 0: 3407.2, 1: 3407.6. Samples: 14125606. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:47:51,683][45164] Avg episode reward: [(0, '6430.742'), (1, '4710.120')] [2023-09-19 10:47:55,847][45899] Updated weights for policy 0, policy_version 14296 (0.0016) [2023-09-19 10:47:55,848][45898] Updated weights for policy 1, policy_version 13840 (0.0015) [2023-09-19 10:47:56,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 14405632. Throughput: 0: 3432.9, 1: 3433.0. Samples: 14167244. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:47:56,683][45164] Avg episode reward: [(0, '6252.256'), (1, '4582.593')] [2023-09-19 10:47:56,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014296_7319552.pth... [2023-09-19 10:47:56,695][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013840_7086080.pth... [2023-09-19 10:47:56,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013640_6983680.pth [2023-09-19 10:47:56,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014096_7217152.pth [2023-09-19 10:48:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 14438400. Throughput: 0: 3468.1, 1: 3468.0. Samples: 14208688. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:48:01,683][45164] Avg episode reward: [(0, '6211.527'), (1, '4659.558')] [2023-09-19 10:48:06,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 14471168. Throughput: 0: 3456.2, 1: 3455.9. Samples: 14229338. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:48:06,683][45164] Avg episode reward: [(0, '6150.139'), (1, '4801.807')] [2023-09-19 10:48:06,685][45811] Saving new best policy, reward=4801.807! [2023-09-19 10:48:08,253][45898] Updated weights for policy 1, policy_version 13920 (0.0013) [2023-09-19 10:48:08,253][45899] Updated weights for policy 0, policy_version 14376 (0.0014) [2023-09-19 10:48:11,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6886.8). Total num frames: 14512128. Throughput: 0: 3451.6, 1: 3451.8. Samples: 14269146. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:48:11,683][45164] Avg episode reward: [(0, '5887.281'), (1, '4788.345')] [2023-09-19 10:48:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014400_7372800.pth... [2023-09-19 10:48:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013944_7139328.pth... [2023-09-19 10:48:11,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014192_7266304.pth [2023-09-19 10:48:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013736_7032832.pth [2023-09-19 10:48:16,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6831.3). Total num frames: 14536704. Throughput: 0: 3421.3, 1: 3421.5. Samples: 14308330. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:48:16,682][45164] Avg episode reward: [(0, '5770.238'), (1, '4792.965')] [2023-09-19 10:48:20,076][45898] Updated weights for policy 1, policy_version 14000 (0.0012) [2023-09-19 10:48:20,078][45899] Updated weights for policy 0, policy_version 14456 (0.0015) [2023-09-19 10:48:21,681][45164] Fps is (10 sec: 6553.8, 60 sec: 6963.2, 300 sec: 6886.8). Total num frames: 14577664. Throughput: 0: 3451.2, 1: 3451.3. Samples: 14328516. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:48:21,682][45164] Avg episode reward: [(0, '5590.413'), (1, '4923.854')] [2023-09-19 10:48:21,683][45811] Saving new best policy, reward=4923.854! [2023-09-19 10:48:26,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 14610432. Throughput: 0: 3450.9, 1: 3450.5. Samples: 14370704. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:48:26,682][45164] Avg episode reward: [(0, '5579.428'), (1, '4762.094')] [2023-09-19 10:48:26,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014496_7421952.pth... [2023-09-19 10:48:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014040_7188480.pth... [2023-09-19 10:48:26,693][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014296_7319552.pth [2023-09-19 10:48:26,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013840_7086080.pth [2023-09-19 10:48:31,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 14643200. Throughput: 0: 3402.2, 1: 3400.9. Samples: 14409732. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:48:31,683][45164] Avg episode reward: [(0, '5676.524'), (1, '4743.806')] [2023-09-19 10:48:32,254][45898] Updated weights for policy 1, policy_version 14080 (0.0011) [2023-09-19 10:48:32,255][45899] Updated weights for policy 0, policy_version 14536 (0.0012) [2023-09-19 10:48:36,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 14675968. Throughput: 0: 3410.7, 1: 3410.9. Samples: 14432576. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:48:36,682][45164] Avg episode reward: [(0, '5971.120'), (1, '4842.850')] [2023-09-19 10:48:41,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 14708736. Throughput: 0: 3412.4, 1: 3412.3. Samples: 14474352. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:48:41,683][45164] Avg episode reward: [(0, '6115.121'), (1, '4860.496')] [2023-09-19 10:48:41,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014600_7475200.pth... [2023-09-19 10:48:41,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014144_7241728.pth... [2023-09-19 10:48:41,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014400_7372800.pth [2023-09-19 10:48:41,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000013944_7139328.pth [2023-09-19 10:48:44,636][45898] Updated weights for policy 1, policy_version 14160 (0.0013) [2023-09-19 10:48:44,637][45899] Updated weights for policy 0, policy_version 14616 (0.0017) [2023-09-19 10:48:46,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6690.2, 300 sec: 6831.3). Total num frames: 14741504. Throughput: 0: 3335.1, 1: 3335.4. Samples: 14508858. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:48:46,682][45164] Avg episode reward: [(0, '6302.093'), (1, '4974.236')] [2023-09-19 10:48:46,682][45811] Saving new best policy, reward=4974.236! [2023-09-19 10:48:51,681][45164] Fps is (10 sec: 7373.0, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 14782464. Throughput: 0: 3369.8, 1: 3369.6. Samples: 14532608. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:48:51,682][45164] Avg episode reward: [(0, '6303.682'), (1, '5002.921')] [2023-09-19 10:48:51,683][45811] Saving new best policy, reward=5002.921! [2023-09-19 10:48:55,980][45899] Updated weights for policy 0, policy_version 14696 (0.0015) [2023-09-19 10:48:55,980][45898] Updated weights for policy 1, policy_version 14240 (0.0013) [2023-09-19 10:48:56,682][45164] Fps is (10 sec: 7372.6, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 14815232. Throughput: 0: 3408.5, 1: 3408.1. Samples: 14575894. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:48:56,683][45164] Avg episode reward: [(0, '6354.206'), (1, '4909.504')] [2023-09-19 10:48:56,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014696_7524352.pth... [2023-09-19 10:48:56,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014240_7290880.pth... [2023-09-19 10:48:56,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014040_7188480.pth [2023-09-19 10:48:56,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014496_7421952.pth [2023-09-19 10:49:01,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 14848000. Throughput: 0: 3462.8, 1: 3462.7. Samples: 14619976. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:49:01,683][45164] Avg episode reward: [(0, '6355.061'), (1, '4966.025')] [2023-09-19 10:49:06,681][45164] Fps is (10 sec: 6553.8, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 14880768. Throughput: 0: 3425.5, 1: 3425.4. Samples: 14636806. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:49:06,682][45164] Avg episode reward: [(0, '6146.900'), (1, '4848.829')] [2023-09-19 10:49:08,588][45898] Updated weights for policy 1, policy_version 14320 (0.0016) [2023-09-19 10:49:08,588][45899] Updated weights for policy 0, policy_version 14776 (0.0016) [2023-09-19 10:49:11,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6690.2, 300 sec: 6859.1). Total num frames: 14913536. Throughput: 0: 3358.7, 1: 3359.2. Samples: 14673008. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:49:11,682][45164] Avg episode reward: [(0, '6030.060'), (1, '4770.198')] [2023-09-19 10:49:11,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014792_7573504.pth... [2023-09-19 10:49:11,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014336_7340032.pth... [2023-09-19 10:49:11,695][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014600_7475200.pth [2023-09-19 10:49:11,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014144_7241728.pth [2023-09-19 10:49:16,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6831.3). Total num frames: 14946304. Throughput: 0: 3392.3, 1: 3393.6. Samples: 14715100. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:49:16,683][45164] Avg episode reward: [(0, '6115.019'), (1, '4937.425')] [2023-09-19 10:49:20,015][45898] Updated weights for policy 1, policy_version 14400 (0.0014) [2023-09-19 10:49:20,016][45899] Updated weights for policy 0, policy_version 14856 (0.0012) [2023-09-19 10:49:21,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 14987264. Throughput: 0: 3401.2, 1: 3401.2. Samples: 14738688. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:49:21,683][45164] Avg episode reward: [(0, '5919.329'), (1, '4957.948')] [2023-09-19 10:49:26,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6831.3). Total num frames: 15020032. Throughput: 0: 3419.2, 1: 3419.2. Samples: 14782080. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:49:26,682][45164] Avg episode reward: [(0, '5945.200'), (1, '4900.163')] [2023-09-19 10:49:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014896_7626752.pth... [2023-09-19 10:49:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014440_7393280.pth... [2023-09-19 10:49:26,696][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014696_7524352.pth [2023-09-19 10:49:26,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014240_7290880.pth [2023-09-19 10:49:31,315][45898] Updated weights for policy 1, policy_version 14480 (0.0009) [2023-09-19 10:49:31,316][45899] Updated weights for policy 0, policy_version 14936 (0.0014) [2023-09-19 10:49:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6859.1). Total num frames: 15060992. Throughput: 0: 3539.1, 1: 3538.2. Samples: 14827334. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:49:31,682][45164] Avg episode reward: [(0, '6245.665'), (1, '5102.374')] [2023-09-19 10:49:31,684][45811] Saving new best policy, reward=5102.374! [2023-09-19 10:49:36,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6831.3). Total num frames: 15093760. Throughput: 0: 3491.1, 1: 3492.0. Samples: 14846850. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:49:36,682][45164] Avg episode reward: [(0, '6431.336'), (1, '5292.350')] [2023-09-19 10:49:36,683][45811] Saving new best policy, reward=5292.350! [2023-09-19 10:49:41,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6803.5). Total num frames: 15126528. Throughput: 0: 3471.7, 1: 3472.0. Samples: 14888360. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:49:41,683][45164] Avg episode reward: [(0, '6585.580'), (1, '5277.423')] [2023-09-19 10:49:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015000_7680000.pth... [2023-09-19 10:49:41,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014544_7446528.pth... [2023-09-19 10:49:41,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014336_7340032.pth [2023-09-19 10:49:41,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014792_7573504.pth [2023-09-19 10:49:43,009][45899] Updated weights for policy 0, policy_version 15016 (0.0014) [2023-09-19 10:49:43,009][45898] Updated weights for policy 1, policy_version 14560 (0.0012) [2023-09-19 10:49:46,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6831.3). Total num frames: 15167488. Throughput: 0: 3475.9, 1: 3476.2. Samples: 14932822. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:49:46,682][45164] Avg episode reward: [(0, '6699.910'), (1, '5414.033')] [2023-09-19 10:49:46,684][45810] Saving new best policy, reward=6699.910! [2023-09-19 10:49:46,684][45811] Saving new best policy, reward=5414.033! [2023-09-19 10:49:51,681][45164] Fps is (10 sec: 7373.0, 60 sec: 6963.2, 300 sec: 6803.5). Total num frames: 15200256. Throughput: 0: 3550.6, 1: 3550.9. Samples: 14956374. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:49:51,682][45164] Avg episode reward: [(0, '6815.890'), (1, '5427.125')] [2023-09-19 10:49:51,683][45810] Saving new best policy, reward=6815.890! [2023-09-19 10:49:51,683][45811] Saving new best policy, reward=5427.125! [2023-09-19 10:49:54,307][45899] Updated weights for policy 0, policy_version 15096 (0.0013) [2023-09-19 10:49:54,308][45898] Updated weights for policy 1, policy_version 14640 (0.0014) [2023-09-19 10:49:56,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.8, 300 sec: 6831.3). Total num frames: 15241216. Throughput: 0: 3613.3, 1: 3613.2. Samples: 14998202. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:49:56,682][45164] Avg episode reward: [(0, '6980.434'), (1, '5466.409')] [2023-09-19 10:49:56,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015112_7737344.pth... [2023-09-19 10:49:56,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014656_7503872.pth... [2023-09-19 10:49:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014440_7393280.pth [2023-09-19 10:49:56,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000014896_7626752.pth [2023-09-19 10:49:56,700][45811] Saving new best policy, reward=5466.409! [2023-09-19 10:49:56,700][45810] Saving new best policy, reward=6980.434! [2023-09-19 10:50:01,681][45164] Fps is (10 sec: 7372.6, 60 sec: 7099.7, 300 sec: 6831.3). Total num frames: 15273984. Throughput: 0: 3613.0, 1: 3612.0. Samples: 15040222. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:01,683][45164] Avg episode reward: [(0, '6890.774'), (1, '5407.810')] [2023-09-19 10:50:06,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6963.2, 300 sec: 6775.8). Total num frames: 15298560. Throughput: 0: 3534.9, 1: 3533.9. Samples: 15056782. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:06,683][45164] Avg episode reward: [(0, '6707.713'), (1, '5387.511')] [2023-09-19 10:50:06,791][45899] Updated weights for policy 0, policy_version 15176 (0.0017) [2023-09-19 10:50:06,792][45898] Updated weights for policy 1, policy_version 14720 (0.0015) [2023-09-19 10:50:11,682][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 6803.5). Total num frames: 15339520. Throughput: 0: 3509.9, 1: 3508.7. Samples: 15097920. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:11,683][45164] Avg episode reward: [(0, '6682.589'), (1, '5251.282')] [2023-09-19 10:50:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015208_7786496.pth... [2023-09-19 10:50:11,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014752_7553024.pth... [2023-09-19 10:50:11,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015000_7680000.pth [2023-09-19 10:50:11,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014544_7446528.pth [2023-09-19 10:50:16,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.8, 300 sec: 6803.5). Total num frames: 15372288. Throughput: 0: 3467.7, 1: 3468.5. Samples: 15139460. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:16,682][45164] Avg episode reward: [(0, '6630.751'), (1, '5212.426')] [2023-09-19 10:50:18,405][45898] Updated weights for policy 1, policy_version 14800 (0.0015) [2023-09-19 10:50:18,405][45899] Updated weights for policy 0, policy_version 15256 (0.0014) [2023-09-19 10:50:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6775.8). Total num frames: 15405056. Throughput: 0: 3493.9, 1: 3494.5. Samples: 15161330. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:21,682][45164] Avg episode reward: [(0, '6555.733'), (1, '5177.802')] [2023-09-19 10:50:26,682][45164] Fps is (10 sec: 7372.6, 60 sec: 7099.7, 300 sec: 6803.5). Total num frames: 15446016. Throughput: 0: 3504.4, 1: 3504.4. Samples: 15203754. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:26,683][45164] Avg episode reward: [(0, '6598.472'), (1, '5109.681')] [2023-09-19 10:50:26,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014856_7606272.pth... [2023-09-19 10:50:26,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015312_7839744.pth... [2023-09-19 10:50:26,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014656_7503872.pth [2023-09-19 10:50:26,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015112_7737344.pth [2023-09-19 10:50:29,614][45899] Updated weights for policy 0, policy_version 15336 (0.0013) [2023-09-19 10:50:29,615][45898] Updated weights for policy 1, policy_version 14880 (0.0015) [2023-09-19 10:50:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6803.5). Total num frames: 15478784. Throughput: 0: 3501.7, 1: 3501.6. Samples: 15247970. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:31,682][45164] Avg episode reward: [(0, '6692.664'), (1, '5078.890')] [2023-09-19 10:50:36,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6775.8). Total num frames: 15511552. Throughput: 0: 3464.0, 1: 3463.6. Samples: 15268120. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:36,682][45164] Avg episode reward: [(0, '6738.719'), (1, '4986.125')] [2023-09-19 10:50:41,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6775.8). Total num frames: 15544320. Throughput: 0: 3395.9, 1: 3395.4. Samples: 15303814. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:41,683][45164] Avg episode reward: [(0, '6809.538'), (1, '4921.591')] [2023-09-19 10:50:41,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014952_7655424.pth... [2023-09-19 10:50:41,694][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015408_7888896.pth... [2023-09-19 10:50:41,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014752_7553024.pth [2023-09-19 10:50:41,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015208_7786496.pth [2023-09-19 10:50:42,782][45899] Updated weights for policy 0, policy_version 15416 (0.0016) [2023-09-19 10:50:42,783][45898] Updated weights for policy 1, policy_version 14960 (0.0015) [2023-09-19 10:50:46,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6690.1, 300 sec: 6720.2). Total num frames: 15568896. Throughput: 0: 3316.5, 1: 3317.5. Samples: 15338750. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:46,682][45164] Avg episode reward: [(0, '6751.767'), (1, '4991.833')] [2023-09-19 10:50:51,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.6, 300 sec: 6775.8). Total num frames: 15609856. Throughput: 0: 3393.7, 1: 3394.6. Samples: 15362256. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:51,683][45164] Avg episode reward: [(0, '6826.681'), (1, '5021.654')] [2023-09-19 10:50:54,370][45898] Updated weights for policy 1, policy_version 15040 (0.0015) [2023-09-19 10:50:54,370][45899] Updated weights for policy 0, policy_version 15496 (0.0015) [2023-09-19 10:50:56,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6690.1, 300 sec: 6775.8). Total num frames: 15642624. Throughput: 0: 3437.0, 1: 3438.4. Samples: 15407314. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:50:56,682][45164] Avg episode reward: [(0, '6928.612'), (1, '5105.366')] [2023-09-19 10:50:56,698][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015512_7942144.pth... [2023-09-19 10:50:56,702][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015056_7708672.pth... [2023-09-19 10:50:56,705][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015312_7839744.pth [2023-09-19 10:50:56,706][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014856_7606272.pth [2023-09-19 10:51:01,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6775.8). Total num frames: 15683584. Throughput: 0: 3449.5, 1: 3448.8. Samples: 15449888. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:51:01,683][45164] Avg episode reward: [(0, '7005.033'), (1, '5259.891')] [2023-09-19 10:51:01,684][45810] Saving new best policy, reward=7005.033! [2023-09-19 10:51:05,724][45899] Updated weights for policy 0, policy_version 15576 (0.0015) [2023-09-19 10:51:05,725][45898] Updated weights for policy 1, policy_version 15120 (0.0012) [2023-09-19 10:51:06,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6775.8). Total num frames: 15716352. Throughput: 0: 3445.1, 1: 3444.9. Samples: 15471380. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:51:06,682][45164] Avg episode reward: [(0, '6984.140'), (1, '5385.140')] [2023-09-19 10:51:11,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6803.5). Total num frames: 15749120. Throughput: 0: 3424.2, 1: 3424.4. Samples: 15511938. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:51:11,682][45164] Avg episode reward: [(0, '7041.364'), (1, '5371.256')] [2023-09-19 10:51:11,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015608_7991296.pth... [2023-09-19 10:51:11,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015152_7757824.pth... [2023-09-19 10:51:11,693][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015408_7888896.pth [2023-09-19 10:51:11,693][45810] Saving new best policy, reward=7041.364! [2023-09-19 10:51:11,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000014952_7655424.pth [2023-09-19 10:51:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6803.5). Total num frames: 15781888. Throughput: 0: 3341.7, 1: 3341.5. Samples: 15548714. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:51:16,682][45164] Avg episode reward: [(0, '7075.626'), (1, '5348.400')] [2023-09-19 10:51:16,683][45810] Saving new best policy, reward=7075.626! [2023-09-19 10:51:18,544][45899] Updated weights for policy 0, policy_version 15656 (0.0011) [2023-09-19 10:51:18,545][45898] Updated weights for policy 1, policy_version 15200 (0.0014) [2023-09-19 10:51:21,681][45164] Fps is (10 sec: 6963.3, 60 sec: 6895.0, 300 sec: 6845.2). Total num frames: 15818752. Throughput: 0: 3352.7, 1: 3352.9. Samples: 15569872. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:51:21,682][45164] Avg episode reward: [(0, '7035.531'), (1, '5397.964')] [2023-09-19 10:51:26,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 15855616. Throughput: 0: 3446.2, 1: 3445.6. Samples: 15613946. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:51:26,683][45164] Avg episode reward: [(0, '6961.061'), (1, '5478.630')] [2023-09-19 10:51:26,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015712_8044544.pth... [2023-09-19 10:51:26,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015256_7811072.pth... [2023-09-19 10:51:26,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015512_7942144.pth [2023-09-19 10:51:26,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015056_7708672.pth [2023-09-19 10:51:26,705][45811] Saving new best policy, reward=5478.630! [2023-09-19 10:51:29,472][45899] Updated weights for policy 0, policy_version 15736 (0.0013) [2023-09-19 10:51:29,473][45898] Updated weights for policy 1, policy_version 15280 (0.0013) [2023-09-19 10:51:31,681][45164] Fps is (10 sec: 7782.2, 60 sec: 6963.2, 300 sec: 6886.8). Total num frames: 15896576. Throughput: 0: 3569.2, 1: 3569.2. Samples: 15659982. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:51:31,682][45164] Avg episode reward: [(0, '6745.738'), (1, '5439.031')] [2023-09-19 10:51:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6859.1). Total num frames: 15929344. Throughput: 0: 3560.9, 1: 3560.9. Samples: 15682740. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:51:36,682][45164] Avg episode reward: [(0, '6325.333'), (1, '5455.981')] [2023-09-19 10:51:40,515][45898] Updated weights for policy 1, policy_version 15360 (0.0015) [2023-09-19 10:51:40,515][45899] Updated weights for policy 0, policy_version 15816 (0.0013) [2023-09-19 10:51:41,682][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6859.1). Total num frames: 15970304. Throughput: 0: 3550.6, 1: 3550.4. Samples: 15726860. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:51:41,683][45164] Avg episode reward: [(0, '6098.837'), (1, '5529.355')] [2023-09-19 10:51:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015368_7868416.pth... [2023-09-19 10:51:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015824_8101888.pth... [2023-09-19 10:51:41,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015152_7757824.pth [2023-09-19 10:51:41,698][45811] Saving new best policy, reward=5529.355! [2023-09-19 10:51:41,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015608_7991296.pth [2023-09-19 10:51:46,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7236.3, 300 sec: 6859.1). Total num frames: 16003072. Throughput: 0: 3521.4, 1: 3522.3. Samples: 15766856. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:51:46,683][45164] Avg episode reward: [(0, '5788.931'), (1, '5535.059')] [2023-09-19 10:51:46,684][45811] Saving new best policy, reward=5535.059! [2023-09-19 10:51:51,688][45164] Fps is (10 sec: 6549.4, 60 sec: 7099.0, 300 sec: 6886.7). Total num frames: 16035840. Throughput: 0: 3510.6, 1: 3510.6. Samples: 15787380. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:51:51,691][45164] Avg episode reward: [(0, '5780.299'), (1, '5481.124')] [2023-09-19 10:51:53,000][45899] Updated weights for policy 0, policy_version 15896 (0.0012) [2023-09-19 10:51:53,000][45898] Updated weights for policy 1, policy_version 15440 (0.0014) [2023-09-19 10:51:56,681][45164] Fps is (10 sec: 6553.7, 60 sec: 7099.7, 300 sec: 6886.8). Total num frames: 16068608. Throughput: 0: 3498.0, 1: 3497.1. Samples: 15826716. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:51:56,682][45164] Avg episode reward: [(0, '5843.338'), (1, '5486.931')] [2023-09-19 10:51:56,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015920_8151040.pth... [2023-09-19 10:51:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015464_7917568.pth... [2023-09-19 10:51:56,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015712_8044544.pth [2023-09-19 10:51:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015256_7811072.pth [2023-09-19 10:52:01,682][45164] Fps is (10 sec: 6557.8, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 16101376. Throughput: 0: 3592.1, 1: 3592.2. Samples: 15872010. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:52:01,683][45164] Avg episode reward: [(0, '6013.382'), (1, '5541.702')] [2023-09-19 10:52:01,684][45811] Saving new best policy, reward=5541.702! [2023-09-19 10:52:04,362][45899] Updated weights for policy 0, policy_version 15976 (0.0014) [2023-09-19 10:52:04,363][45898] Updated weights for policy 1, policy_version 15520 (0.0014) [2023-09-19 10:52:06,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 16134144. Throughput: 0: 3571.0, 1: 3571.2. Samples: 15891272. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:52:06,682][45164] Avg episode reward: [(0, '6249.252'), (1, '5615.206')] [2023-09-19 10:52:06,683][45811] Saving new best policy, reward=5615.206! [2023-09-19 10:52:11,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 16166912. Throughput: 0: 3531.3, 1: 3532.1. Samples: 15931798. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:52:11,683][45164] Avg episode reward: [(0, '6369.825'), (1, '5699.515')] [2023-09-19 10:52:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016016_8200192.pth... [2023-09-19 10:52:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015560_7966720.pth... [2023-09-19 10:52:11,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015824_8101888.pth [2023-09-19 10:52:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015368_7868416.pth [2023-09-19 10:52:11,701][45811] Saving new best policy, reward=5699.515! [2023-09-19 10:52:16,682][45164] Fps is (10 sec: 6553.4, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 16199680. Throughput: 0: 3411.0, 1: 3410.8. Samples: 15966964. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:52:16,683][45164] Avg episode reward: [(0, '6387.747'), (1, '5526.408')] [2023-09-19 10:52:17,151][45898] Updated weights for policy 1, policy_version 15600 (0.0014) [2023-09-19 10:52:17,151][45899] Updated weights for policy 0, policy_version 16056 (0.0013) [2023-09-19 10:52:21,681][45164] Fps is (10 sec: 7373.2, 60 sec: 7031.5, 300 sec: 6914.6). Total num frames: 16240640. Throughput: 0: 3410.8, 1: 3410.9. Samples: 15989716. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:52:21,682][45164] Avg episode reward: [(0, '6573.749'), (1, '5518.580')] [2023-09-19 10:52:26,681][45164] Fps is (10 sec: 7373.0, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 16273408. Throughput: 0: 3389.0, 1: 3387.8. Samples: 16031816. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:52:26,682][45164] Avg episode reward: [(0, '6604.223'), (1, '5652.104')] [2023-09-19 10:52:26,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016120_8253440.pth... [2023-09-19 10:52:26,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015664_8019968.pth... [2023-09-19 10:52:26,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000015920_8151040.pth [2023-09-19 10:52:26,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015464_7917568.pth [2023-09-19 10:52:28,276][45899] Updated weights for policy 0, policy_version 16136 (0.0012) [2023-09-19 10:52:28,277][45898] Updated weights for policy 1, policy_version 15680 (0.0013) [2023-09-19 10:52:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 16314368. Throughput: 0: 3470.7, 1: 3470.8. Samples: 16079222. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:52:31,682][45164] Avg episode reward: [(0, '6507.078'), (1, '5479.858')] [2023-09-19 10:52:36,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 16347136. Throughput: 0: 3467.5, 1: 3467.3. Samples: 16099400. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:52:36,683][45164] Avg episode reward: [(0, '6389.607'), (1, '5581.884')] [2023-09-19 10:52:39,417][45898] Updated weights for policy 1, policy_version 15760 (0.0014) [2023-09-19 10:52:39,417][45899] Updated weights for policy 0, policy_version 16216 (0.0015) [2023-09-19 10:52:41,682][45164] Fps is (10 sec: 6553.3, 60 sec: 6826.6, 300 sec: 6914.6). Total num frames: 16379904. Throughput: 0: 3528.4, 1: 3529.2. Samples: 16144308. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:52:41,684][45164] Avg episode reward: [(0, '6377.645'), (1, '5601.503')] [2023-09-19 10:52:41,722][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015776_8077312.pth... [2023-09-19 10:52:41,724][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016232_8310784.pth... [2023-09-19 10:52:41,725][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015560_7966720.pth [2023-09-19 10:52:41,730][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016016_8200192.pth [2023-09-19 10:52:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 16420864. Throughput: 0: 3481.1, 1: 3481.2. Samples: 16185310. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:52:46,682][45164] Avg episode reward: [(0, '6357.786'), (1, '5732.114')] [2023-09-19 10:52:46,684][45811] Saving new best policy, reward=5732.114! [2023-09-19 10:52:51,222][45898] Updated weights for policy 1, policy_version 15840 (0.0013) [2023-09-19 10:52:51,222][45899] Updated weights for policy 0, policy_version 16296 (0.0017) [2023-09-19 10:52:51,681][45164] Fps is (10 sec: 7373.1, 60 sec: 6964.0, 300 sec: 6942.4). Total num frames: 16453632. Throughput: 0: 3479.6, 1: 3479.5. Samples: 16204432. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:52:51,683][45164] Avg episode reward: [(0, '6360.983'), (1, '5590.364')] [2023-09-19 10:52:56,682][45164] Fps is (10 sec: 5734.3, 60 sec: 6826.6, 300 sec: 6914.6). Total num frames: 16478208. Throughput: 0: 3476.1, 1: 3475.5. Samples: 16244622. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:52:56,683][45164] Avg episode reward: [(0, '6475.863'), (1, '5688.289')] [2023-09-19 10:52:56,707][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015872_8126464.pth... [2023-09-19 10:52:56,708][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016328_8359936.pth... [2023-09-19 10:52:56,711][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015664_8019968.pth [2023-09-19 10:52:56,714][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016120_8253440.pth [2023-09-19 10:53:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 16519168. Throughput: 0: 3525.7, 1: 3525.7. Samples: 16284280. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:53:01,683][45164] Avg episode reward: [(0, '6626.361'), (1, '5695.515')] [2023-09-19 10:53:03,998][45899] Updated weights for policy 0, policy_version 16376 (0.0010) [2023-09-19 10:53:03,999][45898] Updated weights for policy 1, policy_version 15920 (0.0015) [2023-09-19 10:53:06,681][45164] Fps is (10 sec: 7373.0, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 16551936. Throughput: 0: 3482.2, 1: 3482.0. Samples: 16303108. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:53:06,682][45164] Avg episode reward: [(0, '6509.743'), (1, '5604.876')] [2023-09-19 10:53:11,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 16584704. Throughput: 0: 3489.3, 1: 3490.4. Samples: 16345904. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:53:11,682][45164] Avg episode reward: [(0, '6525.292'), (1, '5617.980')] [2023-09-19 10:53:11,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015968_8175616.pth... [2023-09-19 10:53:11,689][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016424_8409088.pth... [2023-09-19 10:53:11,695][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015776_8077312.pth [2023-09-19 10:53:11,696][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016232_8310784.pth [2023-09-19 10:53:15,448][45898] Updated weights for policy 1, policy_version 16000 (0.0013) [2023-09-19 10:53:15,449][45899] Updated weights for policy 0, policy_version 16456 (0.0011) [2023-09-19 10:53:16,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.8, 300 sec: 6942.4). Total num frames: 16625664. Throughput: 0: 3450.8, 1: 3450.7. Samples: 16389788. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:53:16,682][45164] Avg episode reward: [(0, '6546.051'), (1, '5636.318')] [2023-09-19 10:53:21,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 16658432. Throughput: 0: 3453.0, 1: 3453.2. Samples: 16410176. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:53:21,682][45164] Avg episode reward: [(0, '6470.023'), (1, '5481.394')] [2023-09-19 10:53:26,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 16691200. Throughput: 0: 3420.6, 1: 3420.5. Samples: 16452156. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:53:26,683][45164] Avg episode reward: [(0, '6418.344'), (1, '5518.764')] [2023-09-19 10:53:26,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016072_8228864.pth... [2023-09-19 10:53:26,694][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016528_8462336.pth... [2023-09-19 10:53:26,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015872_8126464.pth [2023-09-19 10:53:26,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016328_8359936.pth [2023-09-19 10:53:26,950][45899] Updated weights for policy 0, policy_version 16536 (0.0014) [2023-09-19 10:53:26,950][45898] Updated weights for policy 1, policy_version 16080 (0.0015) [2023-09-19 10:53:31,681][45164] Fps is (10 sec: 7372.6, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 16732160. Throughput: 0: 3482.6, 1: 3481.3. Samples: 16498688. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:53:31,683][45164] Avg episode reward: [(0, '6588.557'), (1, '5584.036')] [2023-09-19 10:53:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 16764928. Throughput: 0: 3520.6, 1: 3520.4. Samples: 16521280. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:53:36,683][45164] Avg episode reward: [(0, '6435.991'), (1, '5594.942')] [2023-09-19 10:53:37,779][45899] Updated weights for policy 0, policy_version 16616 (0.0011) [2023-09-19 10:53:37,780][45898] Updated weights for policy 1, policy_version 16160 (0.0014) [2023-09-19 10:53:41,681][45164] Fps is (10 sec: 7372.9, 60 sec: 7099.8, 300 sec: 6997.9). Total num frames: 16805888. Throughput: 0: 3566.5, 1: 3567.4. Samples: 16565646. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:53:41,682][45164] Avg episode reward: [(0, '6341.704'), (1, '5575.384')] [2023-09-19 10:53:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016640_8519680.pth... [2023-09-19 10:53:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016184_8286208.pth... [2023-09-19 10:53:41,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016424_8409088.pth [2023-09-19 10:53:41,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000015968_8175616.pth [2023-09-19 10:53:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 16838656. Throughput: 0: 3566.9, 1: 3566.0. Samples: 16605264. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:53:46,682][45164] Avg episode reward: [(0, '6364.532'), (1, '5629.407')] [2023-09-19 10:53:49,497][45898] Updated weights for policy 1, policy_version 16240 (0.0015) [2023-09-19 10:53:49,497][45899] Updated weights for policy 0, policy_version 16696 (0.0013) [2023-09-19 10:53:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 16871424. Throughput: 0: 3603.0, 1: 3603.3. Samples: 16627392. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:53:51,682][45164] Avg episode reward: [(0, '6463.904'), (1, '5569.629')] [2023-09-19 10:53:56,682][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 6970.1). Total num frames: 16904192. Throughput: 0: 3517.0, 1: 3516.0. Samples: 16662390. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:53:56,683][45164] Avg episode reward: [(0, '6499.685'), (1, '5564.380')] [2023-09-19 10:53:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016736_8568832.pth... [2023-09-19 10:53:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016280_8335360.pth... [2023-09-19 10:53:56,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016528_8462336.pth [2023-09-19 10:53:56,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016072_8228864.pth [2023-09-19 10:54:01,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 16936960. Throughput: 0: 3510.7, 1: 3510.5. Samples: 16705738. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:01,682][45164] Avg episode reward: [(0, '6566.134'), (1, '5565.782')] [2023-09-19 10:54:01,964][45899] Updated weights for policy 0, policy_version 16776 (0.0016) [2023-09-19 10:54:01,965][45898] Updated weights for policy 1, policy_version 16320 (0.0014) [2023-09-19 10:54:06,681][45164] Fps is (10 sec: 6553.8, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 16969728. Throughput: 0: 3510.3, 1: 3510.2. Samples: 16726098. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:06,683][45164] Avg episode reward: [(0, '6648.074'), (1, '5647.506')] [2023-09-19 10:54:11,682][45164] Fps is (10 sec: 7372.6, 60 sec: 7099.7, 300 sec: 6997.9). Total num frames: 17010688. Throughput: 0: 3524.8, 1: 3524.9. Samples: 16769394. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:11,683][45164] Avg episode reward: [(0, '6752.687'), (1, '5642.922')] [2023-09-19 10:54:11,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016840_8622080.pth... [2023-09-19 10:54:11,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016384_8388608.pth... [2023-09-19 10:54:11,702][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016184_8286208.pth [2023-09-19 10:54:11,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016640_8519680.pth [2023-09-19 10:54:13,900][45899] Updated weights for policy 0, policy_version 16856 (0.0014) [2023-09-19 10:54:13,901][45898] Updated weights for policy 1, policy_version 16400 (0.0012) [2023-09-19 10:54:16,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 17043456. Throughput: 0: 3442.0, 1: 3443.3. Samples: 16808524. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:16,682][45164] Avg episode reward: [(0, '6747.142'), (1, '5631.482')] [2023-09-19 10:54:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 17076224. Throughput: 0: 3425.8, 1: 3426.0. Samples: 16829608. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:54:21,682][45164] Avg episode reward: [(0, '6628.903'), (1, '5723.293')] [2023-09-19 10:54:25,976][45899] Updated weights for policy 0, policy_version 16936 (0.0014) [2023-09-19 10:54:25,977][45898] Updated weights for policy 1, policy_version 16480 (0.0014) [2023-09-19 10:54:26,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 17108992. Throughput: 0: 3380.3, 1: 3380.1. Samples: 16869868. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:54:26,683][45164] Avg episode reward: [(0, '6509.272'), (1, '5683.049')] [2023-09-19 10:54:26,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016480_8437760.pth... [2023-09-19 10:54:26,694][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016936_8671232.pth... [2023-09-19 10:54:26,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016736_8568832.pth [2023-09-19 10:54:26,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016280_8335360.pth [2023-09-19 10:54:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 17141760. Throughput: 0: 3373.1, 1: 3373.8. Samples: 16908876. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:31,682][45164] Avg episode reward: [(0, '6618.608'), (1, '5655.391')] [2023-09-19 10:54:36,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 17174528. Throughput: 0: 3377.2, 1: 3377.1. Samples: 16931336. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:36,682][45164] Avg episode reward: [(0, '6471.866'), (1, '5696.605')] [2023-09-19 10:54:38,192][45898] Updated weights for policy 1, policy_version 16560 (0.0013) [2023-09-19 10:54:38,193][45899] Updated weights for policy 0, policy_version 17016 (0.0014) [2023-09-19 10:54:41,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 17215488. Throughput: 0: 3447.0, 1: 3448.1. Samples: 16972668. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:41,682][45164] Avg episode reward: [(0, '6483.739'), (1, '5621.173')] [2023-09-19 10:54:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017040_8724480.pth... [2023-09-19 10:54:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016584_8491008.pth... [2023-09-19 10:54:41,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016384_8388608.pth [2023-09-19 10:54:41,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016840_8622080.pth [2023-09-19 10:54:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 17248256. Throughput: 0: 3433.3, 1: 3432.7. Samples: 17014706. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:46,683][45164] Avg episode reward: [(0, '6624.613'), (1, '5728.119')] [2023-09-19 10:54:49,492][45899] Updated weights for policy 0, policy_version 17096 (0.0014) [2023-09-19 10:54:49,493][45898] Updated weights for policy 1, policy_version 16640 (0.0014) [2023-09-19 10:54:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 17289216. Throughput: 0: 3454.6, 1: 3454.8. Samples: 17037018. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:51,682][45164] Avg episode reward: [(0, '6582.871'), (1, '5695.857')] [2023-09-19 10:54:56,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 17321984. Throughput: 0: 3464.9, 1: 3464.7. Samples: 17081228. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:54:56,683][45164] Avg episode reward: [(0, '6711.881'), (1, '5731.876')] [2023-09-19 10:54:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016688_8544256.pth... [2023-09-19 10:54:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017144_8777728.pth... [2023-09-19 10:54:56,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016480_8437760.pth [2023-09-19 10:54:56,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000016936_8671232.pth [2023-09-19 10:55:01,518][45898] Updated weights for policy 1, policy_version 16720 (0.0014) [2023-09-19 10:55:01,518][45899] Updated weights for policy 0, policy_version 17176 (0.0013) [2023-09-19 10:55:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6970.1). Total num frames: 17354752. Throughput: 0: 3462.4, 1: 3462.3. Samples: 17120136. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:01,682][45164] Avg episode reward: [(0, '6774.127'), (1, '5703.283')] [2023-09-19 10:55:06,681][45164] Fps is (10 sec: 5734.5, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 17379328. Throughput: 0: 3424.2, 1: 3422.9. Samples: 17137728. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:06,682][45164] Avg episode reward: [(0, '6809.585'), (1, '5698.630')] [2023-09-19 10:55:11,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6690.1, 300 sec: 6914.6). Total num frames: 17412096. Throughput: 0: 3403.9, 1: 3404.1. Samples: 17176228. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:55:11,683][45164] Avg episode reward: [(0, '6875.661'), (1, '5494.901')] [2023-09-19 10:55:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016776_8589312.pth... [2023-09-19 10:55:11,694][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017232_8822784.pth... [2023-09-19 10:55:11,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016584_8491008.pth [2023-09-19 10:55:11,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017040_8724480.pth [2023-09-19 10:55:14,128][45899] Updated weights for policy 0, policy_version 17256 (0.0011) [2023-09-19 10:55:14,128][45898] Updated weights for policy 1, policy_version 16800 (0.0013) [2023-09-19 10:55:16,681][45164] Fps is (10 sec: 7372.6, 60 sec: 6826.6, 300 sec: 6942.4). Total num frames: 17453056. Throughput: 0: 3449.9, 1: 3450.2. Samples: 17219382. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:55:16,683][45164] Avg episode reward: [(0, '6966.988'), (1, '5504.924')] [2023-09-19 10:55:21,681][45164] Fps is (10 sec: 8192.1, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 17494016. Throughput: 0: 3449.7, 1: 3449.7. Samples: 17241808. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:21,682][45164] Avg episode reward: [(0, '7164.061'), (1, '5378.859')] [2023-09-19 10:55:21,684][45810] Saving new best policy, reward=7164.061! [2023-09-19 10:55:25,408][45898] Updated weights for policy 1, policy_version 16880 (0.0014) [2023-09-19 10:55:25,408][45899] Updated weights for policy 0, policy_version 17336 (0.0013) [2023-09-19 10:55:26,682][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 17526784. Throughput: 0: 3453.8, 1: 3453.9. Samples: 17283514. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:26,682][45164] Avg episode reward: [(0, '7170.325'), (1, '5477.192')] [2023-09-19 10:55:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016888_8646656.pth... [2023-09-19 10:55:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017344_8880128.pth... [2023-09-19 10:55:26,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016688_8544256.pth [2023-09-19 10:55:26,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017144_8777728.pth [2023-09-19 10:55:26,700][45810] Saving new best policy, reward=7170.325! [2023-09-19 10:55:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 17559552. Throughput: 0: 3478.1, 1: 3478.9. Samples: 17327774. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:31,682][45164] Avg episode reward: [(0, '7160.014'), (1, '5600.162')] [2023-09-19 10:55:36,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 17592320. Throughput: 0: 3452.4, 1: 3452.5. Samples: 17347740. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:36,683][45164] Avg episode reward: [(0, '7276.247'), (1, '5659.363')] [2023-09-19 10:55:36,684][45810] Saving new best policy, reward=7276.247! [2023-09-19 10:55:37,191][45899] Updated weights for policy 0, policy_version 17416 (0.0013) [2023-09-19 10:55:37,191][45898] Updated weights for policy 1, policy_version 16960 (0.0015) [2023-09-19 10:55:41,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 17625088. Throughput: 0: 3436.8, 1: 3436.9. Samples: 17390548. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:41,683][45164] Avg episode reward: [(0, '7130.440'), (1, '5686.975')] [2023-09-19 10:55:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016992_8699904.pth... [2023-09-19 10:55:41,694][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017448_8933376.pth... [2023-09-19 10:55:41,695][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016776_8589312.pth [2023-09-19 10:55:41,699][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017232_8822784.pth [2023-09-19 10:55:46,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 17657856. Throughput: 0: 3403.8, 1: 3403.6. Samples: 17426468. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:46,683][45164] Avg episode reward: [(0, '7059.251'), (1, '5904.738')] [2023-09-19 10:55:46,684][45811] Saving new best policy, reward=5904.738! [2023-09-19 10:55:49,148][45898] Updated weights for policy 1, policy_version 17040 (0.0015) [2023-09-19 10:55:49,149][45899] Updated weights for policy 0, policy_version 17496 (0.0014) [2023-09-19 10:55:51,681][45164] Fps is (10 sec: 7373.0, 60 sec: 6826.7, 300 sec: 6970.1). Total num frames: 17698816. Throughput: 0: 3458.8, 1: 3458.9. Samples: 17449026. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:51,682][45164] Avg episode reward: [(0, '6995.411'), (1, '5883.180')] [2023-09-19 10:55:56,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 17731584. Throughput: 0: 3477.7, 1: 3476.9. Samples: 17489180. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:55:56,682][45164] Avg episode reward: [(0, '6933.067'), (1, '5752.960')] [2023-09-19 10:55:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017544_8982528.pth... [2023-09-19 10:55:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017088_8749056.pth... [2023-09-19 10:55:56,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017344_8880128.pth [2023-09-19 10:55:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016888_8646656.pth [2023-09-19 10:56:01,009][45899] Updated weights for policy 0, policy_version 17576 (0.0016) [2023-09-19 10:56:01,010][45898] Updated weights for policy 1, policy_version 17120 (0.0012) [2023-09-19 10:56:01,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6826.7, 300 sec: 6942.4). Total num frames: 17764352. Throughput: 0: 3469.8, 1: 3469.6. Samples: 17531656. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:56:01,682][45164] Avg episode reward: [(0, '6849.845'), (1, '5497.586')] [2023-09-19 10:56:06,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6826.6, 300 sec: 6914.6). Total num frames: 17788928. Throughput: 0: 3393.4, 1: 3392.4. Samples: 17547170. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:56:06,682][45164] Avg episode reward: [(0, '6841.103'), (1, '5505.215')] [2023-09-19 10:56:11,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6942.4). Total num frames: 17829888. Throughput: 0: 3386.4, 1: 3385.0. Samples: 17588226. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:56:11,682][45164] Avg episode reward: [(0, '6853.780'), (1, '5382.911')] [2023-09-19 10:56:11,688][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017640_9031680.pth... [2023-09-19 10:56:11,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017184_8798208.pth... [2023-09-19 10:56:11,694][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017448_8933376.pth [2023-09-19 10:56:11,695][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000016992_8699904.pth [2023-09-19 10:56:13,926][45899] Updated weights for policy 0, policy_version 17656 (0.0012) [2023-09-19 10:56:13,927][45898] Updated weights for policy 1, policy_version 17200 (0.0015) [2023-09-19 10:56:16,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6928.5). Total num frames: 17862656. Throughput: 0: 3351.4, 1: 3351.2. Samples: 17629390. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:56:16,682][45164] Avg episode reward: [(0, '6713.378'), (1, '5229.018')] [2023-09-19 10:56:21,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.2, 300 sec: 6914.6). Total num frames: 17895424. Throughput: 0: 3369.1, 1: 3369.0. Samples: 17650956. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:56:21,682][45164] Avg episode reward: [(0, '6736.269'), (1, '5270.872')] [2023-09-19 10:56:25,548][45898] Updated weights for policy 1, policy_version 17280 (0.0012) [2023-09-19 10:56:25,548][45899] Updated weights for policy 0, policy_version 17736 (0.0012) [2023-09-19 10:56:26,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 17936384. Throughput: 0: 3348.5, 1: 3348.5. Samples: 17691910. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:56:26,682][45164] Avg episode reward: [(0, '6514.043'), (1, '5436.561')] [2023-09-19 10:56:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017744_9084928.pth... [2023-09-19 10:56:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017288_8851456.pth... [2023-09-19 10:56:26,695][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017544_8982528.pth [2023-09-19 10:56:26,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017088_8749056.pth [2023-09-19 10:56:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 17969152. Throughput: 0: 3448.2, 1: 3447.9. Samples: 17736792. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:56:31,683][45164] Avg episode reward: [(0, '6377.777'), (1, '5275.251')] [2023-09-19 10:56:36,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 18001920. Throughput: 0: 3388.4, 1: 3389.6. Samples: 17754038. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:56:36,683][45164] Avg episode reward: [(0, '6366.516'), (1, '5320.074')] [2023-09-19 10:56:37,809][45899] Updated weights for policy 0, policy_version 17816 (0.0013) [2023-09-19 10:56:37,809][45898] Updated weights for policy 1, policy_version 17360 (0.0013) [2023-09-19 10:56:41,682][45164] Fps is (10 sec: 5734.3, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 18026496. Throughput: 0: 3350.2, 1: 3350.7. Samples: 17790724. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:56:41,683][45164] Avg episode reward: [(0, '6435.486'), (1, '5281.740')] [2023-09-19 10:56:41,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017832_9129984.pth... [2023-09-19 10:56:41,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017376_8896512.pth... [2023-09-19 10:56:41,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017640_9031680.pth [2023-09-19 10:56:41,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017184_8798208.pth [2023-09-19 10:56:46,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6690.1, 300 sec: 6859.2). Total num frames: 18059264. Throughput: 0: 3297.4, 1: 3297.6. Samples: 17828430. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:56:46,683][45164] Avg episode reward: [(0, '6559.952'), (1, '5157.885')] [2023-09-19 10:56:50,146][45899] Updated weights for policy 0, policy_version 17896 (0.0013) [2023-09-19 10:56:50,146][45898] Updated weights for policy 1, policy_version 17440 (0.0012) [2023-09-19 10:56:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6690.1, 300 sec: 6886.8). Total num frames: 18100224. Throughput: 0: 3370.9, 1: 3371.9. Samples: 17850596. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:56:51,682][45164] Avg episode reward: [(0, '6538.983'), (1, '5132.217')] [2023-09-19 10:56:56,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6690.1, 300 sec: 6886.8). Total num frames: 18132992. Throughput: 0: 3399.2, 1: 3400.5. Samples: 17894210. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:56:56,682][45164] Avg episode reward: [(0, '6614.976'), (1, '5124.893')] [2023-09-19 10:56:56,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017936_9183232.pth... [2023-09-19 10:56:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017480_8949760.pth... [2023-09-19 10:56:56,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017744_9084928.pth [2023-09-19 10:56:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017288_8851456.pth [2023-09-19 10:57:01,681][45899] Updated weights for policy 0, policy_version 17976 (0.0012) [2023-09-19 10:57:01,681][45898] Updated weights for policy 1, policy_version 17520 (0.0014) [2023-09-19 10:57:01,688][45164] Fps is (10 sec: 7367.7, 60 sec: 6825.9, 300 sec: 6914.4). Total num frames: 18173952. Throughput: 0: 3411.6, 1: 3411.8. Samples: 17936492. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:57:01,690][45164] Avg episode reward: [(0, '6600.928'), (1, '5178.573')] [2023-09-19 10:57:06,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 18206720. Throughput: 0: 3423.7, 1: 3423.7. Samples: 17959092. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:57:06,683][45164] Avg episode reward: [(0, '6652.253'), (1, '5210.653')] [2023-09-19 10:57:11,681][45164] Fps is (10 sec: 6558.2, 60 sec: 6826.7, 300 sec: 6914.6). Total num frames: 18239488. Throughput: 0: 3438.5, 1: 3438.3. Samples: 18001366. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 10:57:11,682][45164] Avg episode reward: [(0, '6676.802'), (1, '5256.806')] [2023-09-19 10:57:11,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018040_9236480.pth... [2023-09-19 10:57:11,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017584_9003008.pth... [2023-09-19 10:57:11,696][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017832_9129984.pth [2023-09-19 10:57:11,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017376_8896512.pth [2023-09-19 10:57:13,237][45899] Updated weights for policy 0, policy_version 18056 (0.0014) [2023-09-19 10:57:13,237][45898] Updated weights for policy 1, policy_version 17600 (0.0011) [2023-09-19 10:57:16,681][45164] Fps is (10 sec: 6963.2, 60 sec: 6894.9, 300 sec: 6900.7). Total num frames: 18276352. Throughput: 0: 3404.9, 1: 3405.2. Samples: 18043248. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:57:16,682][45164] Avg episode reward: [(0, '6588.880'), (1, '5489.793')] [2023-09-19 10:57:21,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6914.6). Total num frames: 18313216. Throughput: 0: 3462.0, 1: 3462.1. Samples: 18065622. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:57:21,683][45164] Avg episode reward: [(0, '6535.421'), (1, '5665.232')] [2023-09-19 10:57:25,405][45899] Updated weights for policy 0, policy_version 18136 (0.0014) [2023-09-19 10:57:25,405][45898] Updated weights for policy 1, policy_version 17680 (0.0014) [2023-09-19 10:57:26,682][45164] Fps is (10 sec: 6143.9, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 18337792. Throughput: 0: 3483.2, 1: 3482.3. Samples: 18104168. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:57:26,683][45164] Avg episode reward: [(0, '6595.658'), (1, '5660.605')] [2023-09-19 10:57:26,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017680_9052160.pth... [2023-09-19 10:57:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018136_9285632.pth... [2023-09-19 10:57:26,697][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017480_8949760.pth [2023-09-19 10:57:26,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000017936_9183232.pth [2023-09-19 10:57:31,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 18378752. Throughput: 0: 3505.0, 1: 3504.9. Samples: 18143878. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:57:31,683][45164] Avg episode reward: [(0, '6680.325'), (1, '5849.067')] [2023-09-19 10:57:36,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 18411520. Throughput: 0: 3498.9, 1: 3498.9. Samples: 18165498. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 10:57:36,683][45164] Avg episode reward: [(0, '6480.647'), (1, '5887.123')] [2023-09-19 10:57:37,164][45899] Updated weights for policy 0, policy_version 18216 (0.0014) [2023-09-19 10:57:37,164][45898] Updated weights for policy 1, policy_version 17760 (0.0015) [2023-09-19 10:57:41,682][45164] Fps is (10 sec: 6553.5, 60 sec: 6963.2, 300 sec: 6859.1). Total num frames: 18444288. Throughput: 0: 3461.1, 1: 3460.9. Samples: 18205700. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:57:41,683][45164] Avg episode reward: [(0, '6545.442'), (1, '5980.221')] [2023-09-19 10:57:41,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018240_9338880.pth... [2023-09-19 10:57:41,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017784_9105408.pth... [2023-09-19 10:57:41,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017584_9003008.pth [2023-09-19 10:57:41,701][45811] Saving new best policy, reward=5980.221! [2023-09-19 10:57:41,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018040_9236480.pth [2023-09-19 10:57:46,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6963.2, 300 sec: 6859.1). Total num frames: 18477056. Throughput: 0: 3431.3, 1: 3431.3. Samples: 18245260. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:57:46,682][45164] Avg episode reward: [(0, '6640.790'), (1, '6003.490')] [2023-09-19 10:57:46,683][45811] Saving new best policy, reward=6003.490! [2023-09-19 10:57:49,724][45898] Updated weights for policy 1, policy_version 17840 (0.0015) [2023-09-19 10:57:49,725][45899] Updated weights for policy 0, policy_version 18296 (0.0015) [2023-09-19 10:57:51,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.7, 300 sec: 6886.8). Total num frames: 18509824. Throughput: 0: 3400.1, 1: 3400.1. Samples: 18265102. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:57:51,682][45164] Avg episode reward: [(0, '6584.619'), (1, '5767.693')] [2023-09-19 10:57:56,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6859.1). Total num frames: 18542592. Throughput: 0: 3360.0, 1: 3360.2. Samples: 18303772. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:57:56,682][45164] Avg episode reward: [(0, '6480.073'), (1, '5893.558')] [2023-09-19 10:57:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018336_9388032.pth... [2023-09-19 10:57:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017880_9154560.pth... [2023-09-19 10:57:56,703][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017680_9052160.pth [2023-09-19 10:57:56,703][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018136_9285632.pth [2023-09-19 10:58:01,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6690.9, 300 sec: 6859.1). Total num frames: 18575360. Throughput: 0: 3331.8, 1: 3331.7. Samples: 18343106. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-09-19 10:58:01,682][45164] Avg episode reward: [(0, '6579.645'), (1, '5931.842')] [2023-09-19 10:58:02,302][45899] Updated weights for policy 0, policy_version 18376 (0.0013) [2023-09-19 10:58:02,303][45898] Updated weights for policy 1, policy_version 17920 (0.0015) [2023-09-19 10:58:06,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6690.1, 300 sec: 6859.1). Total num frames: 18608128. Throughput: 0: 3287.6, 1: 3287.5. Samples: 18361498. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:58:06,682][45164] Avg episode reward: [(0, '6522.590'), (1, '5997.725')] [2023-09-19 10:58:11,682][45164] Fps is (10 sec: 6553.4, 60 sec: 6690.1, 300 sec: 6831.3). Total num frames: 18640896. Throughput: 0: 3343.1, 1: 3344.2. Samples: 18405098. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:58:11,683][45164] Avg episode reward: [(0, '6620.165'), (1, '5988.262')] [2023-09-19 10:58:11,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018432_9437184.pth... [2023-09-19 10:58:11,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017976_9203712.pth... [2023-09-19 10:58:11,700][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017784_9105408.pth [2023-09-19 10:58:11,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018240_9338880.pth [2023-09-19 10:58:14,987][45898] Updated weights for policy 1, policy_version 18000 (0.0014) [2023-09-19 10:58:14,987][45899] Updated weights for policy 0, policy_version 18456 (0.0014) [2023-09-19 10:58:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6621.9, 300 sec: 6831.3). Total num frames: 18673664. Throughput: 0: 3292.2, 1: 3291.4. Samples: 18440140. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:58:16,682][45164] Avg episode reward: [(0, '6758.124'), (1, '6025.874')] [2023-09-19 10:58:16,684][45811] Saving new best policy, reward=6025.874! [2023-09-19 10:58:21,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6831.3). Total num frames: 18706432. Throughput: 0: 3245.7, 1: 3245.8. Samples: 18457614. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:58:21,682][45164] Avg episode reward: [(0, '6702.124'), (1, '6134.223')] [2023-09-19 10:58:21,684][45811] Saving new best policy, reward=6134.223! [2023-09-19 10:58:26,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6690.2, 300 sec: 6803.5). Total num frames: 18739200. Throughput: 0: 3265.2, 1: 3265.2. Samples: 18499566. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:58:26,683][45164] Avg episode reward: [(0, '6731.440'), (1, '6067.868')] [2023-09-19 10:58:26,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018528_9486336.pth... [2023-09-19 10:58:26,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018072_9252864.pth... [2023-09-19 10:58:26,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018336_9388032.pth [2023-09-19 10:58:26,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017880_9154560.pth [2023-09-19 10:58:27,545][45899] Updated weights for policy 0, policy_version 18536 (0.0013) [2023-09-19 10:58:27,546][45898] Updated weights for policy 1, policy_version 18080 (0.0014) [2023-09-19 10:58:31,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6803.5). Total num frames: 18771968. Throughput: 0: 3241.3, 1: 3241.2. Samples: 18536970. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:58:31,682][45164] Avg episode reward: [(0, '6756.046'), (1, '6071.263')] [2023-09-19 10:58:36,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6748.0). Total num frames: 18796544. Throughput: 0: 3220.4, 1: 3219.2. Samples: 18554882. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:58:36,682][45164] Avg episode reward: [(0, '6898.631'), (1, '6059.415')] [2023-09-19 10:58:40,832][45898] Updated weights for policy 1, policy_version 18160 (0.0013) [2023-09-19 10:58:40,832][45899] Updated weights for policy 0, policy_version 18616 (0.0012) [2023-09-19 10:58:41,681][45164] Fps is (10 sec: 5734.3, 60 sec: 6417.1, 300 sec: 6748.0). Total num frames: 18829312. Throughput: 0: 3190.6, 1: 3190.5. Samples: 18590920. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 10:58:41,682][45164] Avg episode reward: [(0, '6935.542'), (1, '5985.420')] [2023-09-19 10:58:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018616_9531392.pth... [2023-09-19 10:58:41,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018160_9297920.pth... [2023-09-19 10:58:41,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000017976_9203712.pth [2023-09-19 10:58:41,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018432_9437184.pth [2023-09-19 10:58:46,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6775.8). Total num frames: 18870272. Throughput: 0: 3252.0, 1: 3252.2. Samples: 18635792. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:58:46,682][45164] Avg episode reward: [(0, '6888.479'), (1, '6032.760')] [2023-09-19 10:58:51,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6775.8). Total num frames: 18903040. Throughput: 0: 3276.1, 1: 3275.9. Samples: 18656338. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:58:51,682][45164] Avg episode reward: [(0, '6999.217'), (1, '6101.019')] [2023-09-19 10:58:52,484][45899] Updated weights for policy 0, policy_version 18696 (0.0016) [2023-09-19 10:58:52,484][45898] Updated weights for policy 1, policy_version 18240 (0.0014) [2023-09-19 10:58:56,681][45164] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6775.8). Total num frames: 18935808. Throughput: 0: 3209.3, 1: 3208.4. Samples: 18693894. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-09-19 10:58:56,683][45164] Avg episode reward: [(0, '7057.449'), (1, '6135.893')] [2023-09-19 10:58:56,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018264_9351168.pth... [2023-09-19 10:58:56,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018720_9584640.pth... [2023-09-19 10:58:56,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018072_9252864.pth [2023-09-19 10:58:56,700][45811] Saving new best policy, reward=6135.893! [2023-09-19 10:58:56,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018528_9486336.pth [2023-09-19 10:59:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6775.8). Total num frames: 18968576. Throughput: 0: 3254.6, 1: 3255.3. Samples: 18733088. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:59:01,682][45164] Avg episode reward: [(0, '7099.585'), (1, '6086.716')] [2023-09-19 10:59:06,198][45898] Updated weights for policy 1, policy_version 18320 (0.0012) [2023-09-19 10:59:06,198][45899] Updated weights for policy 0, policy_version 18776 (0.0015) [2023-09-19 10:59:06,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6720.2). Total num frames: 18993152. Throughput: 0: 3242.5, 1: 3242.2. Samples: 18749428. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:59:06,683][45164] Avg episode reward: [(0, '6973.369'), (1, '6076.216')] [2023-09-19 10:59:11,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6720.2). Total num frames: 19025920. Throughput: 0: 3165.0, 1: 3164.8. Samples: 18784410. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:59:11,682][45164] Avg episode reward: [(0, '6831.567'), (1, '6226.298')] [2023-09-19 10:59:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018808_9629696.pth... [2023-09-19 10:59:11,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018352_9396224.pth... [2023-09-19 10:59:11,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018160_9297920.pth [2023-09-19 10:59:11,699][45811] Saving new best policy, reward=6226.298! [2023-09-19 10:59:11,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018616_9531392.pth [2023-09-19 10:59:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6720.2). Total num frames: 19058688. Throughput: 0: 3212.2, 1: 3212.1. Samples: 18826062. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:59:16,682][45164] Avg episode reward: [(0, '6855.429'), (1, '6207.255')] [2023-09-19 10:59:18,339][45899] Updated weights for policy 0, policy_version 18856 (0.0014) [2023-09-19 10:59:18,340][45898] Updated weights for policy 1, policy_version 18400 (0.0013) [2023-09-19 10:59:21,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6720.2). Total num frames: 19091456. Throughput: 0: 3260.8, 1: 3262.1. Samples: 18848416. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:59:21,682][45164] Avg episode reward: [(0, '6877.072'), (1, '6259.702')] [2023-09-19 10:59:21,722][45811] Saving new best policy, reward=6259.702! [2023-09-19 10:59:26,682][45164] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6720.2). Total num frames: 19124224. Throughput: 0: 3323.8, 1: 3323.2. Samples: 18890038. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 10:59:26,682][45164] Avg episode reward: [(0, '6873.240'), (1, '6209.525')] [2023-09-19 10:59:26,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018904_9678848.pth... [2023-09-19 10:59:26,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018448_9445376.pth... [2023-09-19 10:59:26,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018264_9351168.pth [2023-09-19 10:59:26,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018720_9584640.pth [2023-09-19 10:59:30,276][45898] Updated weights for policy 1, policy_version 18480 (0.0012) [2023-09-19 10:59:30,277][45899] Updated weights for policy 0, policy_version 18936 (0.0013) [2023-09-19 10:59:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6748.0). Total num frames: 19165184. Throughput: 0: 3271.1, 1: 3270.9. Samples: 18930184. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:59:31,682][45164] Avg episode reward: [(0, '6971.440'), (1, '6248.085')] [2023-09-19 10:59:36,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6692.4). Total num frames: 19189760. Throughput: 0: 3236.7, 1: 3235.9. Samples: 18947606. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:59:36,682][45164] Avg episode reward: [(0, '6956.864'), (1, '6274.746')] [2023-09-19 10:59:36,684][45811] Saving new best policy, reward=6274.746! [2023-09-19 10:59:41,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6553.6, 300 sec: 6692.4). Total num frames: 19222528. Throughput: 0: 3245.1, 1: 3246.1. Samples: 18985998. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:59:41,683][45164] Avg episode reward: [(0, '6905.815'), (1, '6154.178')] [2023-09-19 10:59:41,695][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019000_9728000.pth... [2023-09-19 10:59:41,695][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018544_9494528.pth... [2023-09-19 10:59:41,704][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018352_9396224.pth [2023-09-19 10:59:41,704][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018808_9629696.pth [2023-09-19 10:59:43,498][45899] Updated weights for policy 0, policy_version 19016 (0.0012) [2023-09-19 10:59:43,498][45898] Updated weights for policy 1, policy_version 18560 (0.0013) [2023-09-19 10:59:46,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6664.7). Total num frames: 19255296. Throughput: 0: 3213.0, 1: 3212.9. Samples: 19022254. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:59:46,683][45164] Avg episode reward: [(0, '6965.566'), (1, '6197.431')] [2023-09-19 10:59:51,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6664.7). Total num frames: 19288064. Throughput: 0: 3277.6, 1: 3277.8. Samples: 19044422. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-09-19 10:59:51,682][45164] Avg episode reward: [(0, '6917.078'), (1, '6206.625')] [2023-09-19 10:59:55,233][45898] Updated weights for policy 1, policy_version 18640 (0.0014) [2023-09-19 10:59:55,234][45899] Updated weights for policy 0, policy_version 19096 (0.0014) [2023-09-19 10:59:56,682][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6692.4). Total num frames: 19329024. Throughput: 0: 3365.6, 1: 3365.3. Samples: 19087300. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 10:59:56,683][45164] Avg episode reward: [(0, '6909.704'), (1, '6149.296')] [2023-09-19 10:59:56,692][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019104_9781248.pth... [2023-09-19 10:59:56,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018648_9547776.pth... [2023-09-19 10:59:56,702][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000018904_9678848.pth [2023-09-19 10:59:56,702][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018448_9445376.pth [2023-09-19 11:00:01,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6720.2). Total num frames: 19361792. Throughput: 0: 3382.0, 1: 3382.2. Samples: 19130452. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:00:01,683][45164] Avg episode reward: [(0, '6967.750'), (1, '6018.396')] [2023-09-19 11:00:06,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6690.1, 300 sec: 6720.2). Total num frames: 19394560. Throughput: 0: 3352.2, 1: 3352.1. Samples: 19150110. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:00:06,683][45164] Avg episode reward: [(0, '7007.758'), (1, '6118.856')] [2023-09-19 11:00:06,793][45899] Updated weights for policy 0, policy_version 19176 (0.0016) [2023-09-19 11:00:06,794][45898] Updated weights for policy 1, policy_version 18720 (0.0016) [2023-09-19 11:00:11,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6720.2). Total num frames: 19435520. Throughput: 0: 3381.5, 1: 3382.2. Samples: 19194406. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 11:00:11,682][45164] Avg episode reward: [(0, '6903.151'), (1, '6175.581')] [2023-09-19 11:00:11,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018752_9601024.pth... [2023-09-19 11:00:11,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019208_9834496.pth... [2023-09-19 11:00:11,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018544_9494528.pth [2023-09-19 11:00:11,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019000_9728000.pth [2023-09-19 11:00:16,681][45164] Fps is (10 sec: 8192.1, 60 sec: 6963.2, 300 sec: 6720.2). Total num frames: 19476480. Throughput: 0: 3435.6, 1: 3435.8. Samples: 19239398. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-09-19 11:00:16,683][45164] Avg episode reward: [(0, '6897.208'), (1, '6066.604')] [2023-09-19 11:00:17,706][45898] Updated weights for policy 1, policy_version 18800 (0.0013) [2023-09-19 11:00:17,707][45899] Updated weights for policy 0, policy_version 19256 (0.0013) [2023-09-19 11:00:21,681][45164] Fps is (10 sec: 7372.9, 60 sec: 6963.2, 300 sec: 6720.2). Total num frames: 19509248. Throughput: 0: 3487.6, 1: 3488.3. Samples: 19261518. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 11:00:21,682][45164] Avg episode reward: [(0, '6909.364'), (1, '5982.661')] [2023-09-19 11:00:26,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6963.2, 300 sec: 6720.2). Total num frames: 19542016. Throughput: 0: 3517.5, 1: 3517.4. Samples: 19302564. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 11:00:26,682][45164] Avg episode reward: [(0, '6914.785'), (1, '6011.518')] [2023-09-19 11:00:26,686][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019312_9887744.pth... [2023-09-19 11:00:26,687][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018856_9654272.pth... [2023-09-19 11:00:26,691][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019104_9781248.pth [2023-09-19 11:00:26,696][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018648_9547776.pth [2023-09-19 11:00:29,249][45898] Updated weights for policy 1, policy_version 18880 (0.0013) [2023-09-19 11:00:29,250][45899] Updated weights for policy 0, policy_version 19336 (0.0012) [2023-09-19 11:00:31,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6963.2, 300 sec: 6748.0). Total num frames: 19582976. Throughput: 0: 3620.8, 1: 3621.0. Samples: 19348134. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-09-19 11:00:31,682][45164] Avg episode reward: [(0, '7046.186'), (1, '5856.902')] [2023-09-19 11:00:36,681][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.8, 300 sec: 6748.0). Total num frames: 19615744. Throughput: 0: 3578.4, 1: 3578.5. Samples: 19366482. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:00:36,682][45164] Avg episode reward: [(0, '7011.290'), (1, '5828.338')] [2023-09-19 11:00:41,573][45899] Updated weights for policy 0, policy_version 19416 (0.0010) [2023-09-19 11:00:41,574][45898] Updated weights for policy 1, policy_version 18960 (0.0014) [2023-09-19 11:00:41,682][45164] Fps is (10 sec: 6553.6, 60 sec: 7099.7, 300 sec: 6748.0). Total num frames: 19648512. Throughput: 0: 3548.7, 1: 3548.3. Samples: 19406664. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:00:41,683][45164] Avg episode reward: [(0, '7001.560'), (1, '5812.319')] [2023-09-19 11:00:41,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019416_9940992.pth... [2023-09-19 11:00:41,692][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018960_9707520.pth... [2023-09-19 11:00:41,698][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019208_9834496.pth [2023-09-19 11:00:41,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018752_9601024.pth [2023-09-19 11:00:46,681][45164] Fps is (10 sec: 6553.5, 60 sec: 7099.7, 300 sec: 6720.2). Total num frames: 19681280. Throughput: 0: 3524.7, 1: 3524.0. Samples: 19447642. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:00:46,683][45164] Avg episode reward: [(0, '7094.665'), (1, '6093.370')] [2023-09-19 11:00:51,681][45164] Fps is (10 sec: 6553.6, 60 sec: 7099.7, 300 sec: 6720.2). Total num frames: 19714048. Throughput: 0: 3558.0, 1: 3558.0. Samples: 19470330. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 11:00:51,683][45164] Avg episode reward: [(0, '6902.627'), (1, '6133.892')] [2023-09-19 11:00:52,831][45899] Updated weights for policy 0, policy_version 19496 (0.0011) [2023-09-19 11:00:52,832][45898] Updated weights for policy 1, policy_version 19040 (0.0013) [2023-09-19 11:00:56,682][45164] Fps is (10 sec: 7372.7, 60 sec: 7099.7, 300 sec: 6748.0). Total num frames: 19755008. Throughput: 0: 3542.9, 1: 3541.7. Samples: 19513212. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 11:00:56,683][45164] Avg episode reward: [(0, '6860.497'), (1, '6037.173')] [2023-09-19 11:00:56,691][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019064_9760768.pth... [2023-09-19 11:00:56,691][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019520_9994240.pth... [2023-09-19 11:00:56,695][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018856_9654272.pth [2023-09-19 11:00:56,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019312_9887744.pth [2023-09-19 11:01:01,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6775.8). Total num frames: 19787776. Throughput: 0: 3520.4, 1: 3520.3. Samples: 19556232. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-09-19 11:01:01,683][45164] Avg episode reward: [(0, '6860.904'), (1, '6267.435')] [2023-09-19 11:01:04,385][45898] Updated weights for policy 1, policy_version 19120 (0.0014) [2023-09-19 11:01:04,385][45899] Updated weights for policy 0, policy_version 19576 (0.0012) [2023-09-19 11:01:06,681][45164] Fps is (10 sec: 6553.8, 60 sec: 7099.8, 300 sec: 6748.0). Total num frames: 19820544. Throughput: 0: 3515.9, 1: 3516.2. Samples: 19577962. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:01:06,682][45164] Avg episode reward: [(0, '6859.385'), (1, '6264.514')] [2023-09-19 11:01:11,681][45164] Fps is (10 sec: 7372.8, 60 sec: 7099.7, 300 sec: 6775.8). Total num frames: 19861504. Throughput: 0: 3510.4, 1: 3510.7. Samples: 19618514. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:01:11,683][45164] Avg episode reward: [(0, '6899.568'), (1, '6251.534')] [2023-09-19 11:01:11,694][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019168_9814016.pth... [2023-09-19 11:01:11,694][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019624_10047488.pth... [2023-09-19 11:01:11,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019416_9940992.pth [2023-09-19 11:01:11,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000018960_9707520.pth [2023-09-19 11:01:16,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6748.0). Total num frames: 19886080. Throughput: 0: 3413.4, 1: 3413.3. Samples: 19655334. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:01:16,683][45164] Avg episode reward: [(0, '6914.421'), (1, '6235.921')] [2023-09-19 11:01:17,187][45898] Updated weights for policy 1, policy_version 19200 (0.0013) [2023-09-19 11:01:17,188][45899] Updated weights for policy 0, policy_version 19656 (0.0015) [2023-09-19 11:01:21,681][45164] Fps is (10 sec: 5734.4, 60 sec: 6826.6, 300 sec: 6720.2). Total num frames: 19918848. Throughput: 0: 3427.0, 1: 3426.8. Samples: 19674902. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:01:21,683][45164] Avg episode reward: [(0, '6862.815'), (1, '6208.156')] [2023-09-19 11:01:26,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6720.2). Total num frames: 19951616. Throughput: 0: 3431.2, 1: 3432.1. Samples: 19715508. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:01:26,682][45164] Avg episode reward: [(0, '6816.073'), (1, '6028.654')] [2023-09-19 11:01:26,689][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019256_9859072.pth... [2023-09-19 11:01:26,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019712_10092544.pth... [2023-09-19 11:01:26,698][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019064_9760768.pth [2023-09-19 11:01:26,700][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019520_9994240.pth [2023-09-19 11:01:28,866][45899] Updated weights for policy 0, policy_version 19736 (0.0013) [2023-09-19 11:01:28,866][45898] Updated weights for policy 1, policy_version 19280 (0.0012) [2023-09-19 11:01:31,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6826.7, 300 sec: 6748.0). Total num frames: 19992576. Throughput: 0: 3468.8, 1: 3469.4. Samples: 19759864. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 11:01:31,683][45164] Avg episode reward: [(0, '6783.787'), (1, '5993.795')] [2023-09-19 11:01:36,681][45164] Fps is (10 sec: 7372.7, 60 sec: 6826.6, 300 sec: 6775.8). Total num frames: 20025344. Throughput: 0: 3450.0, 1: 3449.8. Samples: 19780824. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 11:01:36,683][45164] Avg episode reward: [(0, '6778.070'), (1, '6010.474')] [2023-09-19 11:01:40,919][45898] Updated weights for policy 1, policy_version 19360 (0.0015) [2023-09-19 11:01:40,919][45899] Updated weights for policy 0, policy_version 19816 (0.0015) [2023-09-19 11:01:41,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6775.8). Total num frames: 20058112. Throughput: 0: 3392.9, 1: 3394.1. Samples: 19818622. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-09-19 11:01:41,682][45164] Avg episode reward: [(0, '6767.309'), (1, '6013.073')] [2023-09-19 11:01:41,690][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019816_10145792.pth... [2023-09-19 11:01:41,690][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019360_9912320.pth... [2023-09-19 11:01:41,697][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019624_10047488.pth [2023-09-19 11:01:41,699][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019168_9814016.pth [2023-09-19 11:01:46,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6775.8). Total num frames: 20099072. Throughput: 0: 3412.4, 1: 3412.8. Samples: 19863366. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:01:46,683][45164] Avg episode reward: [(0, '6730.794'), (1, '5995.355')] [2023-09-19 11:01:51,681][45164] Fps is (10 sec: 7372.8, 60 sec: 6963.2, 300 sec: 6775.8). Total num frames: 20131840. Throughput: 0: 3379.4, 1: 3378.2. Samples: 19882056. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:01:51,683][45164] Avg episode reward: [(0, '6653.886'), (1, '6165.195')] [2023-09-19 11:01:52,407][45898] Updated weights for policy 1, policy_version 19440 (0.0013) [2023-09-19 11:01:52,407][45899] Updated weights for policy 0, policy_version 19896 (0.0013) [2023-09-19 11:01:56,682][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6748.1). Total num frames: 20164608. Throughput: 0: 3406.6, 1: 3406.6. Samples: 19925106. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:01:56,683][45164] Avg episode reward: [(0, '6741.055'), (1, '6116.965')] [2023-09-19 11:01:56,693][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019920_10199040.pth... [2023-09-19 11:01:56,693][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019464_9965568.pth... [2023-09-19 11:01:56,701][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019256_9859072.pth [2023-09-19 11:01:56,701][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019712_10092544.pth [2023-09-19 11:02:01,681][45164] Fps is (10 sec: 6553.6, 60 sec: 6826.7, 300 sec: 6748.0). Total num frames: 20197376. Throughput: 0: 3430.2, 1: 3429.8. Samples: 19964032. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:02:01,683][45164] Avg episode reward: [(0, '6708.190'), (1, '6139.784')] [2023-09-19 11:02:05,051][45899] Updated weights for policy 0, policy_version 19976 (0.0012) [2023-09-19 11:02:05,051][45898] Updated weights for policy 1, policy_version 19520 (0.0016) [2023-09-19 11:02:06,681][45164] Fps is (10 sec: 6553.7, 60 sec: 6826.6, 300 sec: 6748.0). Total num frames: 20230144. Throughput: 0: 3405.5, 1: 3405.5. Samples: 19981396. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-09-19 11:02:06,682][45164] Avg episode reward: [(0, '6784.027'), (1, '6123.434')] [2023-09-19 11:02:08,661][45811] Early stopping after 2 epochs (8 sgd steps), loss delta 0.0000000 [2023-09-19 11:02:08,663][45901] Stopping RolloutWorker_w1... [2023-09-19 11:02:08,663][45902] Stopping RolloutWorker_w2... [2023-09-19 11:02:08,663][45906] Stopping RolloutWorker_w6... [2023-09-19 11:02:08,663][45903] Stopping RolloutWorker_w4... [2023-09-19 11:02:08,663][45901] Loop rollout_proc1_evt_loop terminating... [2023-09-19 11:02:08,663][45902] Loop rollout_proc2_evt_loop terminating... [2023-09-19 11:02:08,663][45164] Component RolloutWorker_w1 stopped! [2023-09-19 11:02:08,663][45810] Stopping Batcher_0... [2023-09-19 11:02:08,663][45900] Stopping RolloutWorker_w0... [2023-09-19 11:02:08,663][45905] Stopping RolloutWorker_w5... [2023-09-19 11:02:08,663][45906] Loop rollout_proc6_evt_loop terminating... [2023-09-19 11:02:08,663][45903] Loop rollout_proc4_evt_loop terminating... [2023-09-19 11:02:08,663][45904] Stopping RolloutWorker_w3... [2023-09-19 11:02:08,663][45900] Loop rollout_proc0_evt_loop terminating... [2023-09-19 11:02:08,663][45164] Component RolloutWorker_w2 stopped! [2023-09-19 11:02:08,663][45810] Loop batcher_evt_loop terminating... [2023-09-19 11:02:08,663][45907] Stopping RolloutWorker_w7... [2023-09-19 11:02:08,664][45164] Component RolloutWorker_w6 stopped! [2023-09-19 11:02:08,663][45905] Loop rollout_proc5_evt_loop terminating... [2023-09-19 11:02:08,664][45164] Component RolloutWorker_w5 stopped! [2023-09-19 11:02:08,664][45164] Component RolloutWorker_w4 stopped! [2023-09-19 11:02:08,664][45904] Loop rollout_proc3_evt_loop terminating... [2023-09-19 11:02:08,664][45164] Component Batcher_0 stopped! [2023-09-19 11:02:08,664][45907] Loop rollout_proc7_evt_loop terminating... [2023-09-19 11:02:08,664][45164] Component RolloutWorker_w0 stopped! [2023-09-19 11:02:08,664][45811] Stopping Batcher_1... [2023-09-19 11:02:08,665][45164] Component RolloutWorker_w3 stopped! [2023-09-19 11:02:08,665][45164] Component RolloutWorker_w7 stopped! [2023-09-19 11:02:08,665][45164] Component Batcher_1 stopped! [2023-09-19 11:02:08,665][45811] Loop batcher_evt_loop terminating... [2023-09-19 11:02:08,666][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019544_10006528.pth... [2023-09-19 11:02:08,670][45810] Early stopping after 2 epochs (8 sgd steps), loss delta 0.0000000 [2023-09-19 11:02:08,671][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000020000_10240000.pth... [2023-09-19 11:02:08,672][45811] Removing ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019360_9912320.pth [2023-09-19 11:02:08,673][45811] Saving ./train_dir/Humanoid/checkpoint_p1/checkpoint_000019544_10006528.pth... [2023-09-19 11:02:08,677][45810] Removing ./train_dir/Humanoid/checkpoint_p0/checkpoint_000019816_10145792.pth [2023-09-19 11:02:08,678][45810] Saving ./train_dir/Humanoid/checkpoint_p0/checkpoint_000020000_10240000.pth... [2023-09-19 11:02:08,678][45811] Stopping LearnerWorker_p1... [2023-09-19 11:02:08,679][45811] Loop learner_proc1_evt_loop terminating... [2023-09-19 11:02:08,679][45164] Component LearnerWorker_p1 stopped! [2023-09-19 11:02:08,682][45810] Stopping LearnerWorker_p0... [2023-09-19 11:02:08,683][45810] Loop learner_proc0_evt_loop terminating... [2023-09-19 11:02:08,683][45164] Component LearnerWorker_p0 stopped! [2023-09-19 11:02:08,731][45899] Weights refcount: 2 0 [2023-09-19 11:02:08,731][45898] Weights refcount: 2 0 [2023-09-19 11:02:08,732][45899] Stopping InferenceWorker_p0-w0... [2023-09-19 11:02:08,732][45898] Stopping InferenceWorker_p1-w0... [2023-09-19 11:02:08,732][45899] Loop inference_proc0-0_evt_loop terminating... [2023-09-19 11:02:08,732][45898] Loop inference_proc1-0_evt_loop terminating... [2023-09-19 11:02:08,732][45164] Component InferenceWorker_p0-w0 stopped! [2023-09-19 11:02:08,733][45164] Component InferenceWorker_p1-w0 stopped! [2023-09-19 11:02:08,734][45164] Waiting for process learner_proc0 to stop... [2023-09-19 11:02:09,405][45164] Waiting for process learner_proc1 to stop... [2023-09-19 11:02:09,406][45164] Waiting for process inference_proc0-0 to join... [2023-09-19 11:02:09,434][45164] Waiting for process inference_proc1-0 to join... [2023-09-19 11:02:09,435][45164] Waiting for process rollout_proc0 to join... [2023-09-19 11:02:09,435][45164] Waiting for process rollout_proc1 to join... [2023-09-19 11:02:09,436][45164] Waiting for process rollout_proc2 to join... [2023-09-19 11:02:09,436][45164] Waiting for process rollout_proc3 to join... [2023-09-19 11:02:09,437][45164] Waiting for process rollout_proc4 to join... [2023-09-19 11:02:09,438][45164] Waiting for process rollout_proc5 to join... [2023-09-19 11:02:09,438][45164] Waiting for process rollout_proc6 to join... [2023-09-19 11:02:09,439][45164] Waiting for process rollout_proc7 to join... [2023-09-19 11:02:09,439][45164] Batcher 0 profile tree view: batching: 47.4322, releasing_batches: 3.5953 [2023-09-19 11:02:09,440][45164] Batcher 1 profile tree view: batching: 47.3755, releasing_batches: 3.5160 [2023-09-19 11:02:09,440][45164] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0052 wait_policy_total: 357.7647 update_model: 35.4030 weight_update: 0.0014 one_step: 0.0013 handle_policy_step: 2325.4037 deserialize: 64.9090, stack: 14.5337, obs_to_device_normalize: 486.5305, forward: 1191.2172, send_messages: 124.2716 prepare_outputs: 307.7446 to_cpu: 162.7287 [2023-09-19 11:02:09,441][45164] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0051 wait_policy_total: 359.4317 update_model: 34.3046 weight_update: 0.0016 one_step: 0.0021 handle_policy_step: 2325.0978 deserialize: 64.7116, stack: 14.4649, obs_to_device_normalize: 485.7064, forward: 1189.5863, send_messages: 124.7010 prepare_outputs: 307.3370 to_cpu: 161.9556 [2023-09-19 11:02:09,442][45164] Learner 0 profile tree view: misc: 0.0149, prepare_batch: 22.5215 train: 111.6281 epoch_init: 0.0634, minibatch_init: 1.7362, losses_postprocess: 2.9450, kl_divergence: 1.3733, after_optimizer: 2.0082 calculate_losses: 32.3278 losses_init: 0.0570, forward_head: 3.6943, bptt_initial: 0.2140, bptt: 0.2144, tail: 12.2034, advantages_returns: 1.6578, losses: 12.3091 update: 68.9213 clip: 8.3698 [2023-09-19 11:02:09,443][45164] Learner 1 profile tree view: misc: 0.0161, prepare_batch: 22.7485 train: 109.7818 epoch_init: 0.0634, minibatch_init: 1.7513, losses_postprocess: 2.9607, kl_divergence: 1.3790, after_optimizer: 2.0198 calculate_losses: 32.5105 losses_init: 0.0590, forward_head: 3.7312, bptt_initial: 0.2229, bptt: 0.2207, tail: 12.3011, advantages_returns: 1.6632, losses: 12.3376 update: 66.8379 clip: 8.4151 [2023-09-19 11:02:09,443][45164] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.4391, enqueue_policy_requests: 68.8110, complete_rollouts: 2.3113, env_step: 1945.1154, overhead: 96.5454 save_policy_outputs: 153.0123 split_output_tensors: 52.0521 [2023-09-19 11:02:09,444][45164] RolloutWorker_w7 profile tree view: wait_for_trajectories: 1.4142, enqueue_policy_requests: 68.4129, complete_rollouts: 2.2325, env_step: 1916.1791, overhead: 95.3861 save_policy_outputs: 151.9001 split_output_tensors: 51.9520 [2023-09-19 11:02:09,445][45164] Loop Runner_EvtLoop terminating... [2023-09-19 11:02:09,446][45164] Runner profile tree view: main_loop: 2904.0075 [2023-09-19 11:02:09,446][45164] Collected {0: 10240000, 1: 10006528}, FPS: 6891.5